How can I create a database programmatically and what is the minimum information I need to do this?
Please no "SQL Server Management Object API " suggestions.
You can either use the SQL Server Management Object API (see task "creating, altering and removing databases"):
var srv = new Server();
var db = new Database(srv, "mydb");
db.Create();
Information on how to get started is here. During SQL server installation you need to install the client SDK, the SMO assemblies are in C:\Program Files\Microsoft SQL Server\100\SDK\Assemblies
Or if you don't want the dependency on these assemblies, you can also simply run DDL statements using ADO.Net (e.g. see this question):
using (var connection = new SqlConnection(myConnectionString))
{
connection.Open();
var command = connection.CreateCommand();
command.CommandText = "CREATE DATABASE mydb";
command.ExecuteNonQuery();
}
Obviously you need a correct connection string: known sql server instance and a user with CREATE DATABASE permission.
Create database 'Databasename'
From the creators:
// your connection string
string connectionString = "Server=(local)\\netsdk;uid=sa;pwd=;database=master";
// your query:
var query = GetDbCreationQuery();
var conn = new SqlConnection(connectionString);
var command = new SqlCommand(query, conn);
try
{
conn.Open();
command.ExecuteNonQuery();
MessageBox.Show("Database is created successfully", "MyProgram",
MessageBoxButtons.OK, MessageBoxIcon.Information);
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
finally
{
if ((conn.State == ConnectionState.Open))
{
conn.Close();
}
}
To create in default location with default settings, just:
static string GetDbCreationQuery()
{
// your db name
string dbName = "MyDatabase";
// db creation query
string query = "CREATE DATABASE " + dbName + ";";
return query;
}
Or, to create it in a specific location:
static string GetDbCreationQuery()
{
// your db name
string dbName = "MyDatabase";
// path to your db files:
// ensure that the directory exists and you have read write permission.
string[] files = { Path.Combine(Application.StartupPath, dbName + ".mdf"),
Path.Combine(Application.StartupPath, dbName + ".ldf") };
// db creation query:
// note that the data file and log file have different logical names
string query = "CREATE DATABASE " + dbName +
" ON PRIMARY" +
" (NAME = " + dbName + "_data," +
" FILENAME = '" + files[0] + "'," +
" SIZE = 3MB," +
" MAXSIZE = 10MB," +
" FILEGROWTH = 10%)" +
" LOG ON" +
" (NAME = " + dbName + "_log," +
" FILENAME = '" + files[1] + "'," +
" SIZE = 1MB," +
" MAXSIZE = 5MB," +
" FILEGROWTH = 10%)" +
";";
return query;
}
Even in case the execution fails, give it another try. The db files might have got created.
You need to open a connection to the server, i.e. you need a server and instance name.
You also need the proper access rights to create a database, so you might need some user name and password depending on the authentication settings on the server.
From the server name and authentication information you can construct a connection string and open a connection.
Then you can use the CREATE DATABASE SQL command (see here on MSDN). The only needed parameter for this command is a database name.
You need connection information: server, possibly instance, a user having create database rights on that server/instance and the corresponding password. Then you can use SMO for creating the database. Here is a small PowerShell example that you can very easily "translate" to C#, for example:
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | Out-Null
$s = New-Object Microsoft.SqlServer.Management.Smo.Server($ServerInstance)
# Instantiate the database object and add the filegroups
$db = New-Object Microsoft.SqlServer.Management.Smo.Database($s, $DatabaseName)
$primaryFG = New-Object Microsoft.SqlServer.Management.Smo.FileGroup($db, 'PRIMARY')
$db.FileGroups.Add($primaryFG)
# Create Data file
$syslogname = $DatabaseName + '_SysData'
$dbdsysfile = New-Object Microsoft.SqlServer.Management.Smo.DataFile($primaryFG, $syslogname)
$primaryFG.Files.Add($dbdsysfile)
$dbdsysfile.FileName = $s.MasterDBPath + '\' + $syslogname + '.mdf'
$dbdsysfile.Size = [double](5.0 * 1024.0)
$dbdsysfile.GrowthType = 'KB'
$dbdsysfile.Growth = 10000.0
$dbdsysfile.IsPrimaryFile = 'True'
# Create Log file
$loglogname = $DatabaseName + '_Log'
$dblfile = New-Object Microsoft.SqlServer.Management.Smo.LogFile($db, $loglogname)
$db.LogFiles.Add($dblfile)
$dblfile.FileName = $s.MasterDBLogPath + '\' + $loglogname + '.ldf'
$dblfile.Size = [double](10.0 * 1024.0)
$dblfile.GrowthType = 'KB'
$dblfile.Growth = 10000.0
# Create database with READ_COMMITTED_SNAPSHOT isolation level.
# Other options can be set on $db object before calling Create.
$db.IsReadCommittedSnapshotOn = $true
$db.RecoveryModel = [Microsoft.SqlServer.Management.Smo.RecoveryModel]::Simple
$db.Create()
Assuming you have the rights to fire off a CREATE DATABASE statement you can do so as you would any other query.
I should stress that being able to do so requires quite high privileges on the server and this would be restricted to DBAs in QA and Production environments.
For that reason I would make sure that your connection uses Windows Integrated Security. That way when the appropriate DBA runs your application the app will function as requested.
Once you have created your database you will also need to fire off the T-SQL to create logins and create users. I'm taking it as obvious that CREATE TABLE/VIEW statements will be needed.
Related
Problem Stement
I am trying to completely automate (via parametrization) my SSIS package. It uses the data flow that reads a .csv file and inserts its contents into SQL Server table. So, I need to do this without using the data flow task.
New setup
I have replaced the data flow task with a script task that does the same thing.
The .csv file is loaded into the DataTable object and then inserted into the destination table using SqlBulkCopy class and SqlConnection instance.
public void Main()
{
var atlas_source_application = (string)Dts.Variables["$Project::Atlas_SourceApplication"].Value;
var ssis_package_name = (string)Dts.Variables["System::PackageName"].Value;
var csv_path = (string)Dts.Variables["$Project::SVM_Directory"].Value;
var atlas_server_name = (string)Dts.Variables["$Project::AtlasProxy_ServerName"].Value;
var atlas_init_catalog_name = (string)Dts.Variables["$Project::AtlasProxy_InitialCatalog"].Value;
var connname = #"Data Source=" + atlas_server_name + ";Initial Catalog=" + atlas_init_catalog_name + ";Integrated Security=SSPI;";
var csv_file_path = #"" + csv_path + "\\" + ssis_package_name + ".csv";
try
{
DataTable csvData = new DataTable();
// Part I - Read
string contents = File.ReadAllText(csv_file_path, System.Text.Encoding.GetEncoding(1252));
TextFieldParser parser = new TextFieldParser(new StringReader(contents));
parser.HasFieldsEnclosedInQuotes = true;
parser.SetDelimiters(",");
string[] fields;
while (!parser.EndOfData)
{
fields = parser.ReadFields();
if (csvData.Columns.Count == 0)
{
foreach (string field in fields)
{
csvData.Columns.Add(new DataColumn(string.IsNullOrWhiteSpace(field.Trim('\"')) ? null : field.Trim('\"'), typeof(string)));
}
}
else
{
csvData.Rows.Add(fields.Select(item => string.IsNullOrWhiteSpace(item.Trim('\"')) ? null : item.Trim('\"')).ToArray());
}
}
parser.Close();
// Part II - Insert
using (SqlConnection dbConnection = new SqlConnection(connname))
{
dbConnection.Open();
using (SqlBulkCopy s = new SqlBulkCopy(dbConnection))
{
s.DestinationTableName = "[" + atlas_source_application + "].[" + ssis_package_name + "]";
foreach (var column in csvData.Columns)
{
s.ColumnMappings.Add(column.ToString(), column.ToString());
}
s.WriteToServer(csvData);
}
}
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception ex)
{
Dts.Events.FireError(0, "Something went wrong ", ex.ToString(), string.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
}
This setup works perfectly fine on my local computer. However, once the package is deployed on the server, the insertion part breaks since the database is nowhere to be found (at least that's what it tells me).
Therefore, I tried to imitate the visual SSIS component inside the data flow task [Destination OLE DB] that uses a connection manager.
Old Setup
OLE DB connection manager setup
OLE DB destination setup
This setup uses OLE DB driver with "SQL Server Native Client 11.0" provider (or simply SQLNCLI11.1), "SSPI" integrated security, "Table or view - fast load" mode of access to data. This setup works perfectly fine locally and on the server.
Desired Setup
Armed with this knowledge I have tried to use OleDbConnection and OleDbCommand classes using this stackoverflow question, but I can't see how to use these components to bulk insert data into the DB.
I have also tried to use the visual SSIS component which is called "Bulk Insert Task", but lo luck there either.
How can I possibly insert in bulk using OLE DB?
I keep getting this exception over and over. I've tried separating my query into two separate queries, that didn't work. I've checked to make sure the db connection is closed elsewhere before it's opened during this method, it's definitely closed before the function is called and opened before any queries.
Below iss the code for the function. I've set breakpoints and the query itself is fine. The code is the exact same that I used previously for updating a PIN function, with just the query string changed, so I don't know why it's causing issues:
Code:
public void transferMoney(string senderIban, decimal senderBalance, string receiverIban, decimal transferAmount,string messageOptional)
{
//myBankAccount.AccountPin = updatedPin;
DataTable dtUser = new DataTable();
sqlconnConnection.Open();
string strQuery2 = #"UPDATE Accounts SET Balance = Balance + " + Convert.ToDecimal(transferAmount) + " WHERE GUID = '" + receiverIban + "';"
+ "UPDATE Accounts SET Balance = Balance - " + Convert.ToDecimal(transferAmount) + " WHERE GUID = '" + senderIban + "';";
// example of a Paramaterised SQL statement.
SQLiteCommand sqlcomCommand2 = new SQLiteCommand(strQuery2, sqlconnConnection);
SQLiteDataAdapter sqldatadptAdapter = new SQLiteDataAdapter(sqlcomCommand2); // local SQL data Adaptor
try
{
// sqldatadptAdapter.Fill(dtUser);
sqlcomCommand2.ExecuteNonQuery();
}
catch (Exception ex)
{
// Exception will the "thrown" when there was a problem
throw new Exception($"UPDATE WAS unsuccessful:\n{ex.Message}");
}
finally
{
sqlconnConnection.Close();
}
Maybe you have a DB browser opened? Or you have accessed the DB some other way. This error only occurs when DB is modified or used elsewhere. If you can't find it, I'd suggest restarting PC just in case there something hidden :)
P.S. Posting this as answer as I cannot comment under the question for technical reasons :)
I'm aware there are similar questions all over the web, but I can't find anything for this particular issue. I have C# experience, but am pretty new to MySQL, so perhaps there's something I'm not understanding. I'm trying to make a simple select in C# from a MySQL table:
string server = "192.168.2.6";
string database = "productintegration";
string uid = "root";
string password = "Password1";
string connectionString = "SERVER=" + server + ";" + "DATABASE=" + database + ";" + "UID=" + uid + ";" + "PASSWORD=" + password + ";";
MySqlConnection connection = new MySqlConnection(connectionString);
string query = "select * from tcdidataimport";
connection.Open();
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataReader dataReader = cmd.ExecuteReader();
DataTable dt = new DataTable();
if (dataReader.HasRows)
{
dt.Load(dataReader);
}
connection.Close();
And I get the following exception:
Authentication to host '192.168.2.6' for user 'root' using method 'mysql_native_password' failed with message: Access denied for user 'root'#'JUSTINSPERSONAL' (using password: YES)
Seems simple enough... Except that I'm already connecting to that server (via MySQL Workbench) using that exact login.
Here is show grants;
Which seems to me that I should be able to log in using root at whatever I want? When I tried to create another user (CDISelector#'%') and grant privileges I got a similar error:
But I'm logged in as root? Am I missing something here? Finally, here's the results of select user(), current_user();
JUSTINSPERSONAL is my PC, 192.168.2.6 is the MySQL machine's IP. Not sure what I'm missing here but this all seems a little strange.
And it turns out I had the password incorrect. Ha.
The Situation: I am creating an automated task which queries MySQL (through ODBC) and inserts the result set to a MS Access Database (.mdb) using OLEDB.
The Code:
OleDbConnection accCon = new OleDbConnection();
OdbcCommand mySQLCon = new OdbcCommand();
try
{
//connect to mysql
Connect();
mySQLCon.Connection = connection;
//connect to access
accCon.ConnectionString = #"Provider=Microsoft.Jet.OLEDB.4.0;" +
#"Data source= " + pathToAccess;
accCon.Open();
var cnt = 0;
while (cnt < 5)
{
if (accCon.State == ConnectionState.Open)
break;
cnt++;
System.Threading.Thread.Sleep(50);
}
if (cnt == 5)
{
ToolBox.logThis("Connection to Access DB did not open. Exit Process");
return;
}
} catch (Exception e)
{
ToolBox.logThis("Faild to Open connections. msg -> " + e.Message + "\\n" + e.StackTrace);
}
OleDbCommand accCmn = new OleDbCommand();
accCmn.Connection = accCon;
//access insert query structure
var insertAccessQuery = "INSERT INTO {0} values({1});";
// key = > tbl name in access, value = > mysql query to b executed
foreach (var table in tblNQuery)
{
try
{
mySQLCon.CommandText = table.Value;
//executed mysql query
using (var dataReader = mySQLCon.ExecuteReader())
{
//variable to hold row data
var rowData = new object[dataReader.FieldCount];
var parameters = "";
//read the result set from mysql query
while (dataReader.Read())
{
//fill rowData with the row values
dataReader.GetValues(rowData);
//build the parameters for insert query
for (var i = 0; i < dataReader.FieldCount; i++)
parameters += "'" + rowData[i] + "',";
parameters = parameters.TrimEnd(',');
//insert to access
accCmn.CommandText = string.Format(insertAccessQuery, table.Key, parameters);
try
{
accCmn.ExecuteNonQuery();
}
catch (Exception exc)
{
ToolBox.logThis("Faild to insert to access db. msg -> " + exc.Message + "\\n\\tInsert query -> " + accCmn.CommandText );
}
parameters = "";
}
}
}
catch (Exception e)
{
ToolBox.logThis("Faild to populate access db. msg -> " + e.Message + "\\n" + e.StackTrace);
}
}
Disconnect();
accCmn.Dispose();
accCon.Close();
The Issues:
The memory usage goes very high (300MB++) while the MS Access file size does not change constantly! Seems like the insert caches the data rather that saving it to disk.
It is very slow! I know my query executes within a few second but rather insertion process takes long.
I have tried using prepared statement in MS Access and insert the values as parameters instead of string concat to create insert query. However I get this exception message:
Data type mismatch in criteria expression.
Anyone know how to fix this or have a better approach?
You could create a VBA macro that uses the DoCmd.TransferDatabase method to pull data through ODBC into your Access database. It would probably be much faster and simpler as well.
To run the VBA code from an external program or scheduled task, simply initiate Access to open your file with the /x command line switch and it will run the import macro on startup. A GB of data though is still going to take a while. I found an article by David Catriel that implemented this approach.
An even better option is to use a different database engine back-end like the free version of SQL Server Express. Then you have a lot more options and it is much more robust. If you need MS Access forms and reports, you can create an ADP project file if you use SQL Server, or you can use linked tables to get at your data. You could even use Access as a front-end to your MySQL database instead of copying all the data if that would satisfy your requirements.
Instead of writing code, you could turn to SQL Server Integration Services (SSIS), and be done before lunch. It is available as an extension to Visual Studio, in case you do not have it on your computer already with SQL Server.
With SSIS you are able to create a reusable SSIS package that can be triggered from the command line or scheduled task. This guide shows how to pull data from MySQL into SQL Server, but the SQL Server part should be easy to replace with Access.
some changes with comment to add transaction for command execution. if transactions is not controlled manually, it will be created and committed every time automatically and it's a time consuming action
OleDbConnection accCon = new OleDbConnection();
OdbcCommand mySQLCon = new OdbcCommand();
try
{
//connect to mysql
Connect();
mySQLCon.Connection = connection;
//connect to access
accCon.ConnectionString = #"Provider=Microsoft.Jet.OLEDB.4.0;" +
#"Data source= " + pathToAccess;
accCon.Open();
var cnt = 0;
while (cnt < 5)
{
if (accCon.State == ConnectionState.Open)
break;
cnt++;
System.Threading.Thread.Sleep(50);
}
if (cnt == 5)
{
ToolBox.logThis("Connection to Access DB did not open. Exit Process");
return;
}
}
catch (Exception e)
{
ToolBox.logThis("Faild to Open connections. msg -> " + e.Message + "\\n" + e.StackTrace);
}
//AMK: transaction starts here
var transaction = accCon.BeginTransaction();
OleDbCommand accCmn = new OleDbCommand();
accCmn.Connection = accCon;
accCmn.Transaction = transaction;
//access insert query structure
var insertAccessQuery = "INSERT INTO {0} values({1});";
// key = > tbl name in access, value = > mysql query to b executed
foreach (var table in tblNQuery)
{
try
{
mySQLCon.CommandText = table.Value;
//executed mysql query
using (var dataReader = mySQLCon.ExecuteReader())
{
//variable to hold row data
var rowData = new object[dataReader.FieldCount];
var parameters = "";
//read the result set from mysql query
while (dataReader.Read())
{
//fill rowData with the row values
dataReader.GetValues(rowData);
//build the parameters for insert query
for (var i = 0; i < dataReader.FieldCount; i++)
parameters += "'" + rowData[i] + "',";
parameters = parameters.TrimEnd(',');
//insert to access
accCmn.CommandText = string.Format(insertAccessQuery, table.Key, parameters);
try
{
accCmn.ExecuteNonQuery();
}
catch (Exception exc)
{
ToolBox.logThis("Faild to insert to access db. msg -> " + exc.Message +
"\\n\\tInsert query -> " + accCmn.CommandText);
}
parameters = "";
}
}
//AMK: transaction commits here if every thing is going well
transaction.Commit();
}
catch (Exception e)
{
ToolBox.logThis("Faild to populate access db. msg -> " + e.Message + "\\n" + e.StackTrace);
//AMK: transaction rollback here if there is a problem
transaction.Rollback();
}
}
Disconnect();
accCmn.Dispose();
accCon.Close();
Create a DSN (data source name) for the SQL server database. Then select that DSN by opening the Microsoft Access database and choosing to import from that DSN. You should have the ability to import that exact 1GB table (schema, data, everything).
More information on using a DSN:
https://support.office.com/en-us/article/Link-to-SQL-Server-data-0474c16d-a473-4458-9cf7-f369b78d3db8
Alternatively you can just link to the SQL server database (not import to an Access table) using that DSN and skip the import altogether.
Should the INSERT be part of a TRANSACTION. Being within a TRANSACTION usually speeds BULK INSERTS
Thanks everyone for the answers. I just found the main problem in my code. The reason for heavy memory usage (issue #1) was ODBC was caching the data from MySQL regardless of C# approach (DataReader). That issue is resolved by checking the Don't cache results of forward-only cursors checkbox in DSN settings. This also made the process slightly faster (30%). However, more concrete approach is still what Brian Pressler and Egil Hansen suggested.But since they require software installation and/or migration plan, easiest way would be to stick to this piece of code.
I'm trying to write simple application in C# which will allow me to backup, zip and send over ftp my SQL Server database.
One problem I have encountered is that I'm not able to create the backup file (.bak) if I try to do it in different location than "C:\Program Files\Microsoft SQL Server\MSSQL.3\MSSQL\Backup" or "C:\Program Files\Microsoft SQL Server\MSSQL.3\MSSQL\Data" folder. I understand that this is a premission problem. Could someone point me to the resources or write here a short snippet how to programmatically add such a permission to any folder on my system.
Regards
Kris
i assume you are running your programm as a scheduled task ... did you give writing permissions to the target folder for the executing user of the task??
edit:
with permissions you can have 2 scenarios:
windows authenification
mixed authentification
if you are using windows authentification, the read and write permissions of the windows user are taken. otherwise the permissions for the sql server service account.
and this behaviour makes sense to me and maybe hits the nail in your scenario!
edit 2:
i don't want to encourage you to do so ... some admins may hate you when you mess up their acl's
but this may do the trick
btw: Magnus Johansson already gave you a "try-this" link
no matter for which method you go - be sure to hand in the correct user (as descriped above!)
(for full history)
...
side-note:
i know this is not the exact answer to your question, but i would recommend you smo to generate backups ...
like
using Microsoft.SqlServer.Management.Smo;
var bdi = new BackupDeviceItem(/* your path inlcuding desired file */);
var backup = new Backup
{
Database = /* name of the database */,
Initialize = true
};
backup.Devices.Add(bdi);
var server = new Server(this.SqlServer);
try
{
backup.SqlBackup(server);
}
catch (Exception ex)
{
// * log or sth
}
you only have to care for the .dll's. take assemblies for the desired server version (some params/properties vary through different server versions)
more info here
Ok Guys, Magnus and dittodhole! Thanks a lot for your help. I have combined Magnus'es link to the article on setting up permisions on the folder together with some more research and finally I've got it :).
So reassuming, I'm using Smo, and to create a folder with proper permissions I have to look for the group instead of win32_Users. Here you go a short snippet if someone finds this post he can find it usefull:
string tempPath = Directory.CreateDirectory("C:\\path_to_your_folder").FullName;
//set permissions
SelectQuery sQuery = new SelectQuery("Win32_Group",
"Domain='" +
System.Environment.UserDomainName.ToString() +
"'");
try
{
DirectoryInfo myDirectoryInfo = new DirectoryInfo("C:\\path_to_your_folder");
DirectorySecurity myDirectorySecurity = myDirectoryInfo.GetAccessControl();
ManagementObjectSearcher mSearcher = new ManagementObjectSearcher(sQuery);
foreach (ManagementObject mObject in mSearcher.Get())
{
string User = System.Environment.UserDomainName + "\\" + mObject["Name"];
if(User.StartsWith("your-machine-name\\SQL"))
{
myDirectorySecurity.
AddAccessRule(new FileSystemAccessRule(User,
FileSystemRights.FullControl,
AccessControlType.Allow));
}
}
myDirectoryInfo.SetAccessControl(myDirectorySecurity);
}
catch (Exception ex)
{
Console.WriteLine(ex.StackTrace);
}
Again thanks everyone for your help! Stackoverflow rocks!
Here is a procedure is use for back up in C#.Hope it helps
public void BackupDatabase (string BackUpLocation, string BackUpFileName, string
DatabaseName, string ServerName )
{
DatabaseName = "[" + DatabaseName + "]";
string fileUNQ = DateTime.Now.Day.ToString() + "_" + DateTime.Now.Month.ToString() + "_" + DateTime.Now.Year.ToString() +"_"+ DateTime.Now.Hour.ToString()+ DateTime.Now .Minute .ToString () + "_" + DateTime .Now .Second .ToString () ;
BackUpFileName = BackUpFileName + fileUNQ + ".bak";
string SQLBackUp = #"BACKUP DATABASE " + DatabaseName + " TO DISK = N'" + BackUpLocation + #"\" + BackUpFileName + #"'";
string svr = "Server=" + ServerName + ";Database=master;Integrated Security=True";
SqlConnection cnBk = new SqlConnection(svr);
SqlCommand cmdBkUp = new SqlCommand(SQLBackUp, cnBk);
try
{
cnBk.Open();
cmdBkUp.ExecuteNonQuery();
Label1.Text = "Done";
Label2.Text = SQLBackUp + " ######## Server name " + ServerName + " Database " + DatabaseName + " successfully backed up to " + BackUpLocation + #"\" + BackUpFileName + "\n Back Up Date : " + DateTime.Now.ToString();
}
catch (Exception ex)
{
Label1.Text = ex.ToString();
Label2.Text = SQLBackUp + " ######## Server name " + ServerName + " Database " + DatabaseName + " successfully backed up to " + BackUpLocation + #"\" + BackUpFileName + "\n Back Up Date : " + DateTime.Now.ToString();
}
finally
{
if (cnBk.State == ConnectionState.Open)
{
cnBk .Close();
}
}
}
Take a look at this article.
Remember to set the permissions for the account that the SQL Server instance is running with.
Although this may not answer your immediate question, I'd advice you to look into SQL Server Integration Services (SSIS). This looks like the exact thing SSIS was created for, and in the 2008 version there's the possibility to use C# code if needed, should the standard components not do what you need (earlier versions used VB.NET).
MSDN SSIS Info Link 1
SSIS 2005 Tutorial Link 2
Take a look at it.