Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.
I'm trying to create a program which will generate a SQL database schema including tables, view, keys, indexes, triggers, etc... like:
CREATE TABLE TableName(....) ....
CREATE VIEW ViewName(...) ....
I know this is possible because SQL Server Management Studio does it (generate script command). However, how does it do it?
UPDATE: I forgot to mention about permissions: I'm an owner of database (in most cases) but I'm not sys-admin. Would there be any difference?
If you are targeting SQL Server only, SMO is very powerful. This is the library that SQL Server Management Studio uses, and contains classes to convert database objects into scripts.
The scripting example here is a great place to start.
for list of tables:
foreach (DataRow row in schemaTbl.Rows)
{
listBox.Items.Add(row["TABLE_NAME"]);
}
for columns from perticular table
object[] objArrRestrict;
objArrRestrict = new object[] {null, null, "Customers", null};
DataTable schemaCols;
schemaCols = con.GetOleDbSchemaTable(OleDbSchemaGuid.Columns, objArrRestrict);
//List the schema info for the selected table
foreach (DataRow row in schemaCols.Rows)
{
listBox.Items.Add(row["COLUMN_NAME"]);
}
Queries of database structure vary by sql server version.
The 2008 R2 page is here, which links to a TABLES page, which links to a sys.tables page, which links to a sys.objects page.
The sys.objects page has relevant samples. This only gets you the table. There are other system objects for column, triggers, views, ...
If you truly mean generate you might want to look into Entity Framework / Code first approach that will essentially boostrap your db for you (Tables, etc).
Related
I must sync 2 tables of two databases one of which is a MSSQL and the other one MySQL. I have to do this through a Windows Service. I currently have created a Windows Service, and what I currently have added to the service is : getting the data from the MySQL database and inserting it into a Data Adapter from which I plan to move the data to the MSSQL database using an insertion through transaction.Can you tell me what is the best approach to this problem and if what I'm doing right now is on the right track, first time I'm doing such a thing.
Well, probably there is a couple ways to solve this. You didn't mention about count and size of your tables, desired synchronization frequency so my answer might be not 100% relevant. Anyway my proposal is:
Each table should have additional column - SyncDate (TIMESTAMP) which by default is null.
Sync logic which is implemented in Windows Service periodically checks each table if there is some data to sync by executing query like this (use pure IDataReader for performance reasons):
SELECT * from TEST_TABLE where SyncDate is null
If above statement returns data then:
collect package of rows to insert (for example 500 rows)
begin transaction on target database and execute insert statement for collected package (set value for column SyncDate to DateTime.Now)
when insert is complete execute statement like this in source db:
UPDATE TEST_TABLE SET SyncDate = #DateTime.Now where ID_PRIMARY_KEY IN (IDENTIFIRES FROM PACKAGE)
commit transaction
collect another packages and repeat algorithm until DataReader provides data
Above algorithm should be applied on both databases
Probably your database has foreign keys so you have to remember that dependant tables should be synchronized in valid order
Wherever it possible you can benefit from multi threading
For this kind of job, SQL Server Integration Services is the way to go.
SSIS is made to Extract, transform, and load data. (ETL process)
You design a worklow with each transformation step of your data, from MySQL to MSSQL.
You then create a job and configure its schedule, and it will be executed by SQL Server.
No need to develop something from scratch and hard to maintain.
If you are already at a point where you are ready to move the MySQL data to SQL. I would suggest you put them on a separate table and issue a Merge command to merge the data. Make sure that you flag back the data that has been updated so you can now get them back and push to MySQL:
MERGE TableFromMySQL AS TARGET USING TableFromSQL AS SOURCE
ON (TARGET.PrimaryKey = SOURCE.PrimaryKey)
WHEN Matched AND (TARGET.Field1 <> SOURCE.Field1
OR TARGET.Field2 <> SOURCE.Field2
OR .. put more field comparison here that you want to sync
THEN
UPDATE
SET TARGET.Field1 = SOURCE.Field1,
TARGET.Field2 = SOURCE.Field2,
//flag the target field as updated
TARGET.IsUpdated = 1,
TARGET.LastUpdatedOn = GETDATE()
WHEN Not Matched
THEN
INSERT(PrimaryKey, Field1, Field2, IsUpdated, LastUpdatedOn)
VALUES(SOURCE.PrimaryKey, SOURCE.Field1, SOURCE.Field2, 1, GETDATE());
At this point, you can now query the MySQL table with IsUpdated = 1 and push all values to MYSQL database. I hope this helps.
If you add the mySql database as a linked server (sp_addlinkedserver) then I think you could do the following in a stored proc on your MS SQL database:
insert table1(col1)
select col1
from openquery('MySqlLinkedDb','select col1 from mySqlDb.dbo.table2')
If you have to do this through a Windows Service, I am curious how you are receiving the data from MySQL? What is initiating the call to the service and what form is the data passed in? Are you getting changes or are you getting a couple refresh of the data?
If you receive a DataTable or some other kind of IEnumerable and its a complete refresh, for example, you could use SqlBulkCopy.WriteToServer(). That would be the fastest and most efficient.
Tom
I need to update a bit field in a table and set this field to true for a specific list of Ids in that table.
The Ids are passed in from an external process.
I guess in pure SQL the most efficient way would be to create a temp table and populate it with the Ids, then join the main table with this and set the bit field accordingly.
I could create a SPROC to take the Ids but there could be 200 - 300,000 rows involved that need this flag set so its probably not the most efficient way. Using the IN statement has limitation wrt the amount of data that can be passed and performance.
How can I achieve the above using the Entity Framework
I guess its possible to create a SPROC to create a temp table but this would not exist from the models perspective.
Is there a way to dynamically add entities at run time. [Or is this approach just going to cause headaches].
I'm making the assumption above though that populating a temp table with 300,000 rows and doing a join would be quicker than calling a SPROC 300,000 times :)
[The Ids are Guids]
Is there another approach that I should consider.
For data volumes like 300k rows, I would forget EF. I would do this by having a table such as:
BatchId RowId
Where RowId is the PK of the row we want to update, and BatchId just refers to this "run" of 300k rows (to allow multiple at once etc).
I would generate a new BatchId (this could be anything unique -Guid leaps to mind), and use SqlBulkCopy to insert te records onto this table, i.e.
100034 17
100034 22
...
100034 134556
I would then use a simgle sproc to do the join and update (and delete the batch from the table).
SqlBulkCopy is the fastest way of getting this volume of data to the server; you won't drown in round-trips. EF is object-oriented : nice for lots of scenarios - but not this one.
I'm assigning Marcs response as the answer but I'd just like to give a little detail on how we implemented the requirement.
Marc response helped greatly in the formulation of our solution.
We had to deal with an aim/guideline to keep within the Entity Framework while not utilizing SPROCS and although our solution may not suit others it has worked for us
We created a Item table in the Database with BatchId [uniqueidentifier] and ItemId varchar columns.
This table was added to the EF model so we did not use temporary tables.
On upload of these Ids this table is populated with the Ids [Inserts are quick enough we find using EF]
We then use context.ExecuteStoreCommand to run the SQL to do join the item table and the main table and update the bit field in the main table for records that exist for the batch Id created specifically for that session.
We finally clear this table for that batchId.
We have the performance, keeping within our no SPROC goal. [Which not of us agree with :) but its a democracy]
Our exact requirements are a little more complex but insofar as needing good update performance using the Entity framework given our specific restrictions it works fine.
Liam
I am developing application VS 2008, .NET 3.5 and I am trying to use LINQ To SQL. I drag & drop tables on the designer to generate the .dbml file.
The problem I have that I have some dynamic tables for search indexing.
I know the structure of table, only the application creates new tables like this:
Files_1_1, Files_1_2, ... Files_m_n
DataSearch_1_1, DataSearch_1_2, DataSearch_m_n
In this case, m and n are integers in the name of the table.
I statically define which columns are available but not the name of table, so I need a way to do this on the fly. Of course, this would also have to include associated tables.
I haven't been able to get good idea about it. I would also be satisfied with just being able to generate LINQ To SQL class for this tables.
Has anyone come across a solution to this problem? I have been looking through blog posts and forums for the past one days in vain. Any sample code is great for me.
Link to sql works with stored procedures and the designer will auto create a class for the return type. You could use dynamic sql in your sp and return linq to sql classes.
You could create a stored procedure like below:
CREATE PROCEDURE spGetFiles
(
#TableName
)
AS
EXEC('SELECT * FROM " + #TableName)
Then in the Visual Studio O/R designer, select the SP from the server explorer and drag it into the designer window in the same way that you add tables. A method with the same name as your SP will be created on your data context class and a class called something like spGetFilesReturnType will be created (i may have got this naming slightly wrong but you get the idea). You then just call the datacontext method with the table name as a string parameter and collections of spGetFilesReturnType objects will be returned.
I have two tables which looks like this:
News: (ID, Title, TagID)
Tags: (ID, Tag)
Each news can only have one tag. What is the most effective way to handle inserts to the news table? The Tags table has like 50 000 rows.
I'm only doing bulk inserts of approx. 300 news at a time, around 2 times per hour. I assume that i need some in-memory cache for the tags?
If the tag is not in the tags table, i need to insert it and set TagID to the newly inserted id.
Hope you'll get the idea!
What version of SQL Server are you using in the background?
If you're using SQL Server 2008, I would recommend bulk-loading the tags and news for each day into a temporary working table, and then using the MERGE statement to update the actual Tags and News table from those working tables. I'd use the C# "SqlBulkCopy" class for that.
MERGE allows you to easily insert only those items that have changed, and possibly update those that already exist, all in one single, handy SQL statement.
If you're on SQL Server 2005 or below, you can do basically the same, but you'll have to write some code (C# or T-SQL) to manually check what needs to be inserted from your temp bulkload tables, and what is already present.
Marc
I presume with each news item you'll get a list of strings that are the supposed "tags". From the structure you've given, you can only have one tag on each news item? That seems unusual, but the below applies anyway.
If your Tags table has an index on it, the searches will be really fast, and the database will take care of the caching anyway, so don't worry about the caching. You'll be amazed how much the database can speed things up when you have indexes in the right place
Do a select from Tags where Tag = whatever1 (do this for each tag), each time if no rows returned insert it, otherwise use the id you've found to do it. Run the proc on each INSERT.
I have a customer that has a SQL database on a hosted server; call the db "myDatabase".
The hosting co. has locked down object explorer - I can't see myDatabase in the database listed (I see tempdb and master). However, if I "use myDatabase" and then "select * from myTable", all works fine.
Since we have no access to object explorer, I can't right click and generate scripts. I thought that I might be able to use SMO to accomplish what I want, but when I attempt something similar to this:
Server myServer = new Server(conn);
Database myDB = server.Databases["myDatabase"];
Table myTbl = myDB.Tables["myTable"];
It fails - myDB is null (when I iterate through the databases collection, as expected, I only see master and tempdb - the db's I can see in object explorer). It obviously has to do with security - if I can't see the table in object explorer, it won't let me access it through SMO. Anyone have any ideas of a workaround or alternate method to allow me to generate a script?
Thx!
I haven't looked at the SMO code, but have you tried using the constructor on the database object? Maybe you can access it directly.
Database myDB = new Database(myServer, "myDatabase");
Is the myDb.Tables collection empty? Could it be that you are referencing it using the wrong name?
One option you could try is to use Linq2Sql to generate a model of the database. You can then use the model to create a new database that should be more or less identical to the original. Look up the DataContext.CreateDatabase method for more info.
Another option would be to list all tables using the following query:
select * from sys.tables
And then listing all columns in the tables using the following:
select * from sys.columns where object_id = (object id from the previous query)
This will give you all tables and columns defined in your database and should be enough to create the database structure. In addition you have system views for other objects defined as well.