For software testing purposes I would like to create a sterile clone (with all data blanked out) of the production database. This way I can run my unit tests on a known set of records every time. I am looking to try and do this programmatically within the unit tests themselves so I can ensure that the tables contain exactly the test data that I need for the functional tests.
I have found the following information relating to creating an Access database within C#. Note: I know Access probably isn't the best solution, but its good enough!
What I would like to know, is there a way of using TableAdapters (perhaps) to replicate the production database schema (without any data) within a blank Access database file?
Do this:
create a copy of the access file; production -> test
connect to test database
enumerate all tables in the database
run DELETE * FROM [table] for all tables. run it several times if you have FK dependencies until there is no error - or TRUNCATE [table] as commented
compact the database
I do not have much experience with Access, but generally you would make a CREATE script for this purpose. Most database tools have a function for creating such a script. Such a script basically is a set of SQL statements that create all the objects (e.g. databases, views).
Searching for CREATE script and Access will give you some starting points.
I have bad experiences with Access as a production database. I won't recommend. Either go with SQLite or Firebird.
Secondly, yes you can use TableAdapters. You need to create two connections for each db. But I think there might be tools available to do this.
Edited **
How big is the database? For up to 4GB, Oracle Express Edition might help. Also, it will be easy to clone from Oracle to Oracle.
Related
I have an c#.net windows based application that uses a database in Microsoft SQL Server 2008. During deployment for very first time to our client(s), we create a copy of our database and deploy it on client(s) remote server along with the UI application. The client database can be on version SQL Server 2005 and higher.
During times the UI application and associated database has gone lots of changes. Since this is a thick client application the client(s) database is not sync with our latest database and unfortunately no one ever made notes of all the changes done. So my challenges are as follows:
How to find any missing columns on database table in Client's Database as compared to my Database? if any?
How to find any missing Primary/Unique Constraints on database table in Client's Database as compared to my Database? if any?
How to find any missing Indexes on database table that exist in Client's Database as compared to my Database? if any?
Please keep in mind the client(s) database size may ranges from 10-100GB, so i cannot plan to just drop all client tables and recreate it.
You can use Data-tier applications. It's built-in feature of SQL Server, so you don't need to use any extra tools.
You can extract data-tier application from your database (in SSMS right-click -> Tasks -> Extract data-tier application) to a DACPAC file, copy the file to the client's server and use it to upgrade the DB there (or generate update script).
It also integrates nicely with SQL Server Data Tools.
For this task, you need a software that compare SQL database. Just like there is a lot of software to compare text, there is a lot to compare database.
Personally, I use AdoptSQLDiff, but there is a bunch. RedGate has developed one also and I know others exists. Just type SQL Database compare in google to find them. You probably can have the job done with the trial period.
These softwares show you which tables was added, deleted or changed. It does the same for views, indexes, triggers, Stored Procedures, User Defined Functions, Constraints. More importantly, those tools generate script to push modifications into the target database. Very handy, but have a look at the script generated, it sometime messes it up by deleting data, but it can be fixed very easily.
There is also the option to compare data in a specific table if you need to.
Here is a screen shot of the interface of another so you know what it's look like.
With SQLServer Management Studio, you can try selecting a database and then Task->Generate Script, selecting appropriate options.
Do the same thing for the 2 db you want to compare. You will get two text files you can compare with a text file software comparer.
Comparison will highlight difference in the db structure.
Not the best way to do it, of course. But it can be a start. If the two dbs are not too different, you should be able to handle the differences
Better option, use some db comparer software. They are meant to compare db structure, constraint indexes and so on. Never used any of them, so cannot give any advice on that
If it is one time thing use any diff tool for DB, VS2010+ has a build in one, allows you to get difference for schema and data in two different files.
If you want to solve problem of your development process, you have wide range of options to implement versioning for data base.
If you are using EF - use Migrations, can't beat that.
If you are only on SQL Server and never looking at other RDBMS, check DAC ( Data-Tier applications, mentioned by Jakub)
Otherwise take a look at more generic solutions, among them I would reccomend you to take a look at DB.UP and if python code is good for you , check Alembic, it allow you to write your migrations using really nice python API.
if nothing works for you, create snapshot of current db schema and start doing differential scripts that you can use with self written tool or DB.UP
I am not sure if this can help, but who knows.
So is there any way to restore the server database on your local environment? If the answer is yes, you can try to join system views for each database and compare them?
I propose something like this(was a quick solution, so please sorry for formatting and other common stuff).
USE [master]
GO
SELECT
LocalDataBaseTable.name AS TableName,
LocalDataBaseTableColumns.name AS [Column],
LocalDataBaseTypes.name AS DataType,
LocalDataBaseTableColumns.max_length,
LocalDataBaseTableColumns.[precision]
INTO #tmpLocalInfo
FROM LocalTable.sys.columns as LocalDataBaseTableColumns
INNER JOIN LocalTable.sys.tables AS LocalDataBaseTable
ON LocalDataBaseTableColumns.object_id = LocalDataBaseTable.object_id
INNER JOIN LocalTable.sys.types AS LocalDataBaseTypes
ON LocalDataBaseTypes.user_type_id = LocalDataBaseTableColumns.user_type_id
SELECT
ServerDataBaseTable.name AS TableName,
ServerDataBaseTableColumns.name AS [Column],
ServerDataBaseTypes.name AS DataType,
ServerDataBaseTableColumns.max_length,
ServerDataBaseTableColumns.[precision]
INTO #tmpServerInfo
FROM ServerTable.sys.columns as ServerDataBaseTableColumns
INNER JOIN ServerTable.sys.tables AS ServerDataBaseTable
ON ServerDataBaseTableColumns.object_id = ServerDataBaseTable.object_id
INNER JOIN ServerTable.sys.types AS ServerDataBaseTypes
ON ServerDataBaseTypes.user_type_id = ServerDataBaseTableColumns.user_type_id
SELECT
#tmpServerInfo.*
FROM #tmpLocalInfo
RIGHT OUTER JOIN #tmpServerInfo
ON #tmpLocalInfo.TableName = #tmpServerInfo.TableName COLLATE DATABASE_DEFAULT
AND #tmpLocalInfo.[Column] = #tmpServerInfo.[Column] COLLATE DATABASE_DEFAULT
WHERE #tmpLocalInfo.[Column] IS NULL
DROP TABLE #tmpLocalInfo
DROP TABLE #tmpServerInfo
This will return all information about missed columns in your local database. The idea is to investigate 'sys' views and to find out if there any suitable solution for you.
You can use this simple script, which show you differences between tables, views, indexes etc.
Compalex is a free lightweight script to compare two database schemas. It
supports MySQL, MS SQL Server and PostgreSQL.
or look at this question Compare two MySQL databases. This question about comparing two MySQL schemas, but some of listed tools supports MSSQL or have MSSQL version (for example http://www.liquibase.org/).
Another answer What is best tool to compare two SQL Server databases (schema and data)?
In my Application I have used Entity Framework Database First approach.
Currently my application is in Dev Environment, now it need to be moved into Test Environment and later into Production Environment.
So is there anyway that I can use .net feature or Entity framework feature to migrate/create database in Test environment. Other than using SQL feature of restoring the database.
Also note that if any enhancement comes then Database structure can change, table schema can change.
So can you suggest me the best way to easily migrate database schema in different environment without losing existing Data.
If you want to take advantage of EF-Migrations feature, you must convert your application to Code First with Existing Database http://msdn.microsoft.com/en-us/data/jj200620.aspx
If you are unable to convert to code first then you must create the update script by hand.
Use a schema compare tool, compare the development and production server.
For each difference found, create an update query.
Once the entire script is finished, test it on the staging server.
Automating the migration is very risky, it depends on the type and size changes you made to the schema. You can't trust any single feature or tool specially if the changes requires data motion (moving data around).
The following links might help you:
How to do Migrations in DB first approach
EF Migrations for Database-first approach?
With Database First, the easiest way to copy a schema is to extract a data tier application in management studio, create an empty database on the target, register it as a data tier application with the same name, and upgrade the empty database using the upgraded file. You can repeat this step to manage schema changes.
Having said that, going forward you're really better off switching your Database First to Code First as it will make change management across your deployments much easier.
Migrations are best way to deal with it
Preferred way to update production db is to first generate sql file and then run the sql file in production environment.
MS had a very good artical on this
http://msdn.microsoft.com/en-in/data/jj591621.aspx#script
I need your help to manage an issue with my C# program. I wrote a GUI that allows the user to manage a lot of data stored in a SQL Server database. Everything works fine but I want to be sure that the application works even when the server is down (for a generic issue).
My plan is to have a local database (e.g. *.mdf database used in Visual Studio) and update it every time the GUI is able to connect to the online SQL Server database.
What do you think? Is there something similar to a procedure or do I need to do it manually (create a .mdf file, check the online version, write the changes etc.)?
Apart from the comments noting that this may not be a good idea (which I agree):
Most of the work must be done manually. If you have a DB model within your application (like when using entity framework) it could be that it can create the DB structure in the local file. Most of your data will need timestamps to determine when they were changed the last time.
The Microsoft Sync Framework might utilize you but I have not used it personally. Look here http://msdn.microsoft.com/en-us/library/bb902854(v=sql.110).aspx
I am in the midst of writing a small program (more to experiment with vs 2010 than anything else)
Despite being an experiment it has some practical use for our local athletics club.
My thought was to access the DB (currently online) to download the current members and store locally on a laptop (this is a MS sql table, used to power the club's website).
Take the laptop to the event (yes there ARE places that don't have internet coverage), add members to that days race (also a row from a sql table (though no changes would be made to this), record results (new records in 3rd table)
Once home, showered and within internet access again, upload/edit the tables as per the race results/member changes etc.
So I was thinking I'd do something like write xml files locally with the data, including a field to indicate changes etc?
If anyone can point me in a direction I would appreciate it...hell if anyone could tell me if this has a name, I'd appreciate it.
Essentially what you need is, in addition to your remote data store, a local data store on your desktop. You could then write your code by hand to sync the data stores when you go offline / online, or you could use the Microsoft Sync framework to handle it for you.
I've personally used the Sync framework on a number of projects and once you get used to the conventions, it's pretty easy to use.
If a local storage format is what your after. SQLite is one option. You can copy your tables from the server to your local SQLite db.
You could also save your data to files, but XML is a horrible format for doing this. You'll probably want to use YAML or JSON instead.
You may want to take a look at SQL Server Compact -- it provides some decent capabilities with synchronizing back with the mothership SQL server.
If you're using MS SQL Server for production, and you only need to work offline on your personal computer, you could install MS SQL Server Express locally. The advantage here over using a different local datastore is that you can reuse your schema, stored procedures, etc. essentially only needing to change the connection string to your application (which you could run locally too through Visual Studio). You would have to write code to manually sync your online and offline db instances, but since it's a small application, it may be reasonable to just copy the entire database from production to local and then from local to production when you get home (assuming you're the only one updating the db, and wouldn't be potentially wiping out any new records entered in production while you were at the event).
Google Gears http://gears.google.com/ is intended if your app is a web app (which I didn't quite get what it is from your description)
When there are a number of people working on a project, all of who could alter the database schema, what's the simplest way to unit test / test / verify it? The main suggestion we've had so far is to write tests for each table to verify column names, constraints, etc.
Has anyone else done anything similar / simpler? We're using C# with SQL Server, if that makes any real difference.
Updates:
The segment of the project we're working on is using SSIS packages to do the bulk of the work so there is very little C# code to write unit tests agains.
The code for creating tables / stored procedures is spread across SQL files. Because of the build system, we could maintain a separate VS DB project file as well, but I'm not sure how that would help us verify the schema either.
One possibly answer is to use Visual Studio for Database developers and keep your schema in source control with the rest of your code. This allows you to see differences and you get a history of who changed what.
Alternatively you could use a tool like SQLCompare to see what has been modified in one database compared to another.
Your (relational) database does two things as far as I'm concerned: 1) Hold data and 2) Hold relations between data.
Holding data is not a behavior so you would not test it
And for ensuring relations just use constraints. Lots of constraints. All over the place.
That is an interesting question! There are lots of tools out there for testing stored procedures but not for testing the database schema.
Don't you find that the unit tests written for code generally find any problems with the database schema?
One approach I have used is to write stored procedures to copy test data from the developer's schema to a test schema. This is pretty rough and ready as the stored procedures generally crash when they come across any differences between the schemas but it does alert you to any changes you haven't been told about.
And nominate someone to be the DBA who monitors changes to the schema?
I've had to do this type of thing before, although not in C#. To begin with, I built a schema migration tool, based on the discussion at Ode to Code (page 1 of 5) (there are also existing tools to do similar things). Importantly, the migration tool I built allowed you to specify the database you were applying the changes to and what version you wanted to apply. Then, following a test first methodology, whenever I needed to make a schema change I would write a test script which would create a test database, apply version changes to the one before my target change script, add some data, apply the change script under test, and confirm that the data was in an expected state.
My main goal with this was to confirm that no data was lost or corrupted during schema migrations, not to check specifically that the schema was in a particular state. A good awareness of your production data set is required, so you can write representative sample data for the tests.
It's debatable if this should be considered unit testing or integration testing. I would tend to consider it integration testing, based on the fact that I don't want to run old tests every time I iterate my code. Whatever you want to call it, I found it to be a useful tool for that situation.
This is an old question but it appears that people are still landing here. So the best tool I have found so far is "SQL Test" by Red Gate. It allows you to create scripts that run as transactions. Allowing you to run "sandboxed" queries for checking the state of the database.
This does not really fit the unit test paradigm. I would suggest version controlling the schema and limiting write access to a single qualified team member such as the DBA or team lead, who can validate any requested changes against the entire application. Schema changes should not be done haphazardly.
Don't you find that the unit tests written for code generally find any problems with the database schema?
This assumes, of course, that your tests test everything.