I am using EntityFramework and code migrations to keep my Postgres database up to date.
Before deployment on a new environment I would like to make sure the database exists and that the user that will execute queries for the application has enough permissions to do this.
So in short I would like to do the following from my Migrations project before running context.Database.Migrate():
Check whether the database exists
If it doesn't exist create it
Create the user if it doesn't exists
Grant permissions to the user
Run migrations
I have two options (I think):
Run multiple statements at once, keeping the logic inside the query itself and avoid having it in code (C#)
Run statements and deciding in code what to do next.
Option 1 has my preference but when I run the statement checking whether the database exists and create it if it doesn't I get an error: CREATE DATABASE cannot be executed from a function.
The query I'm running looks like this:
DO $$
BEGIN
IF NOT EXISTS(SELECT datname FROM pg_database WHERE datname='database_name') THEN
CREATE DATABASE database_name;
END IF;
END
$$
Option 2 would involve running the statement to check if the database exists with the following code:
using (var conn = new NpgsqlConnection("connection"))
{
conn.Open();
var sql = "SELECT 1 FROM pg_database WHERE datname='database_name'";
var command = new NpgsqlCommand(sql, conn);
var dbExists = command.ExecuteScalar();
if(dbExists == null)
{
command = new NpgsqlCommand("CREATE DATABASE database_name", conn);
command.ExecuteNonQuery();
}
conn.Close();
}
The above code works but I think I prefer option 1.
So I have 2 questions:
How can I fix the CREATE DATABASE cannot be executed from a function error.
Is the second option considered 'wrong'?
Related
I have an API in .NET Core 3.1 which insert some data to my databases, the issue is that i had to add a new column to the column where i'm inserting the data, as then the api is called i'm dynamically connect to the database to which i should do the insert, how could i make an altertable if that column not exist and then insert the new item?
Here is the method where i'm inserting the stuff:
public static IActionResult InsertOrder(string piva, string idNegozio, RiepilogoTakeaway ordine, HttpResponse Response) {
string connectionString = getConnectionString(piva);
var query_ordine = #"INSERT INTO `ordini` (`TIPO_OR`, `TAVOLO_OR`, `ORA_OR`, `SENT_OR`, `ID_OR_CFG`, `LOTTERIA_OR`) VALUES (#tipo, #tavolo, NOW(), 0, #id, #lotteria); SELECT LAST_INSERT_ID();";
using var connection = new MySqlConnection(connectionString);
using var cmd = new MySqlCommand(query_ordine, connection);
cmd.Parameters.AddWithValue("#tipo", "STB");
cmd.Parameters.AddWithValue("#tavolo", 0);
cmd.Parameters.AddWithValue("#id", idNegozio);
cmd.Parameters.AddWithValue("#lotteria", ordine.lotteria);
connection.Open();
cmd.Prepare();
string idOrdine = cmd.ExecuteScalar().ToString();
...
}
And that's the alter table i should execute:
ALTER TABLE `ordini`
ADD COLUMN `LOTTERIA_OT` VARCHAR(10) NULL AFTER `ID_OR_CFG`;
Which would be the best way to do it?
Try to deploy the SQL changes before calling it. It's very likely that you'll need higher level of privilege in the database to run the alter or add column, for example, if the SQL user you are using to insert the order doesn't have permissions to change the DB schema.
However, if that is still an option, you will need to execute the SQL schema changeusing command.ExecuteNonQuery().
Possibly worth creating a new method "PrepareSqlTable" so you can call before your query runs. Bear in mind that will impact your application performance/scalability because now we'll be checking that on every single query.
Another alternative, handle this in the catch by looking into the error message and creating it accordingly (provides more performance benefits than the previous option).
Using the OracleClient that comes with ADO.NET in .NET Framework, I'm trying to call OracleCommandBuilder.DeriveParameters() method on a procedure in the database, but I keep getting an OracleException with the message: ORA-06564: object CustOrdersOrders does not exist, even though I created the procedure successfully. I'm more familiar with SQL Server, so perhaps I'm missing something here.
SQL
file 1:
create or replace PACKAGE PKGENTLIB_ARCHITECTURE
IS
TYPE CURENTLIB_ARCHITECTURE IS REF CURSOR;
END PKGENTLIB_ARCHITECTURE;
/
file 2
CREATE OR REPLACE PROCEDURE "CustOrdersOrders"(VCUSTOMERID IN Orders.CustomerID%TYPE := 1, CUR_OUT OUT PKGENTLIB_ARCHITECTURE.CURENTLIB_ARCHITECTURE)
AS
BEGIN
OPEN cur_OUT FOR
SELECT
OrderID,
OrderDate,
RequiredDate,
ShippedDate
FROM Orders
WHERE CustomerID = vCustomerId;
END;
/
Both these files were executed in SQL*Plus as #"path\to\file1.sql".
Code
This is using the Enterprise Library Data Access Application Block, which ultimately wraps the ADO.NET API.
DatabaseProviderFactory factory = new DatabaseProviderFactory(...); //this gets a custom configuration source
Database db = factory.Create("OracleTest");
DbCommand storedProcedure = db.GetStoredProcCommand("CustOrdersOrders");
DbConnection connection = db.CreateConnection();
connection.Open();
storedProcedure.Connection = connection;
db.DiscoverParameters(storedProcedure); //this ultimately calls OracleCommandBuilder.DeriveParameters(), which throws the exception.
When I run direct SQL queries using the same connection, they succeed.
More Details
This is actually part of unit tests written for the Data Access Application Block, which I forked here in an attempt to revive this library. That's why it's using the System.Data.OracleClient and not the ODP.NET. The entire set of tests at https://github.com/tsahi/data-access-application-block/blob/master/source/Tests/Oracle.Tests.VSTS/OracleParameterDiscoveryFixture.cs breaks in a similar way.
The tests are running on an Oracle Database XE I installed locally.
Update
Following question by #madreflection, yes, the following code runs correctly:
Database db = DatabaseFactory.CreateDatabase("OracleTest");
string spName = "AddCountry";
DbCommand dbCommand = db.GetStoredProcCommand(spName);
db.AddInParameter(dbCommand, "vCountryCode", DbType.String);
db.AddInParameter(dbCommand, "vCountryName", DbType.String);
db.SetParameterValue(dbCommand, "vCountryCode", "UK");
db.SetParameterValue(dbCommand, "vCountryName", "United Kingdom");
db.ExecuteNonQuery(dbCommand);
using (DataSet ds = db.ExecuteDataSet(CommandType.Text, "select * from Country where CountryCode='UK'"))
{
Assert.IsTrue(1 == ds.Tables[0].Rows.Count);
Assert.AreEqual("United Kingdom", ds.Tables[0].Rows[0]["CountryName"].ToString().Trim());
}
where "AddCountry" is defined as
CREATE OR REPLACE PROCEDURE ADDCOUNTRY
(vCountryCode IN Country.CountryCode%TYPE,
vCountryName IN Country.CountryName%TYPE
)
AS
BEGIN
INSERT INTO Country (CountryCode,CountryName)
VALUES (vCountryCode,vCountryName);
END;
/
It's interesting to note, though, that in this case the OracleDatabase pointed by db has in it's packages list just EntlibTest, defined (if I understand correctly) by
CREATE OR REPLACE PACKAGE EntlibTest AS
PROCEDURE GetProductDetailsById
(vProductID IN NUMBER,vProductName OUT VARCHAR2,vUnitPrice OUT NUMBER);
END EntlibTest;
/
and then there is another file defining the body of this procedure with
CREATE OR REPLACE PACKAGE BODY EntlibTest AS
PROCEDURE GetProductDetailsById
(vProductID IN NUMBER,vProductName OUT VARCHAR2,vUnitPrice OUT NUMBER)
AS
BEGIN
SELECT ProductName,UnitPrice INTO vProductName,vUnitPrice FROM Products where ProductId = vProductId;
END;
END EntlibTest;
/
I am trying to setup my .NET 4.7.1 program that is connecting to a MySQL database 8.0 to use the minimum privileges to run.
The .NET program is using MySql.Data to make connection. The minimum right for a user to execute a stored procedure is typically only EXECUTE privilege. This works fine from MySQL workbench or command line.
Upon running the .NET program this does return the following exception:
System.Data.SqlTypes.SqlNullValueException: 'Data is Null. This method or property cannot be called on Null values.'
To make it easy, I have create a very small demo program to demonstrate the issue.
Setup of the database:
CREATE DATABASE Spike;
CREATE PROCEDURE TestAccess()
BEGIN
END;
CREATE USER Spike#localhost IDENTIFIED WITH mysql_native_password BY 'sample';
GRANT EXECUTE ON PROCEDURE `TestAccess` TO Spike#localhost;
Setup program code:
static void Main(string[] args)
{
using (MySqlConnection conn = new MySqlConnection("Server=localhost;Database=Spike;uid=Spike;pwd=sample"))
{
conn.Open();
Console.WriteLine("Connection open");
MySqlCommand cmd = new MySqlCommand();
cmd.Connection = conn;
cmd.CommandText = "TestAccess";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.ExecuteNonQuery();
Console.WriteLine("Query executed");
}
Console.ReadKey();
}
The crash happens at the line cmd.ExecuteNonQuery();
The stack from the crash is interesting, since it seems to indicate that the information_schema is queried. When logging all statements I can see that the last statement before the exception is:
SELECT * FROM information_schema.routines WHERE 1=1 AND routine_schema LIKE 'Spike' AND routine_name LIKE 'TestAccess'
I cannot grant different rights on information_schema, but I could give more rights on the stored procedure to make more information visible in the routines table, this feels wrong however. Simple tests with granting CREATE and ALTER access also did not work.
Is there something else I can do, without granting too much privileges?
This appears to be a bug in Connector/NET, similar to bug 75301 but a little different. When it's trying to determine parameter metadata for the procedure, it first creates a MySqlSchemaCollection named Procedures with all metadata about the procedure. (This is the SELECT * FROM information_schema.routines WHERE 1=1 AND routine_schema LIKE 'Spike' AND routine_name LIKE 'TestAccess' query you see in your log.)
The Spike user account doesn't have permission to read the ROUTINE_DEFINITION column, so it is NULL. Connector/NET expects this field to be non-NULL and throws a SqlNullValueException exception trying to read it.
There are two workarounds:
1) The first, which you've discovered, is to set CheckParameters=False in your connection string. This will disable retrieval of stored procedure metadata (avoiding the crash), but may lead to harder-to-debug problems calling other stored procedures if you don't get the order and type of parameters exactly right. (Connector/NET can no longer map them for you using the metadata.)
2) Switch to a different ADO.NET MySQL library that doesn't have this bug: MySqlConnector on NuGet. It's highly compatible with Connector/NET, performs faster, and fixes a lot of known issues.
I found an answer with which I am quite pleased. It is changing the connection string by adding CheckParameters=false:
using (MySqlConnection conn = new MySqlConnection("Server=localhost;Database=Spike;uid=Spike;pwd=sample;CheckParameters=false"))
This disables parameter checking, and thereby information_schema queries.
I am writing a CLR Update trigger for SQL Server 2008 R2. The trigger needs to write updated values to a table in another database hosted in the same SQL Server instance. When I try to open a connection created with the following connection string from within my trigger I get a "SecurityException":
...new SqlConnection("Data Source=(local);Initial Catalog=[my database];Integrated Security=True")
It is highly desirable that I leave my assembly's permission level as SAFE. I am pretty sure that I'd have to set my assembly's permission level to EXTERNAL_ACCESS to connect to a remote database, but is it possible to connect to another database in the same SQL Server instance with the SAFE permission level?
Thanks.
Yes, it is definitely possible to reference another database in a SQLCLR trigger that resides inside of an Assembly marked as SAFE. The error encountered in the question is due simply to using a regular / external connection string, which requires a PERMISSION_SET of EXTERNAL_ACCESS. But using the in-process / internal connection string of "Context Connection = true;" allows you to run whatever query you want, including a query that references another database via 3-part object name.
I was able to do this with the following code:
Main Table (in TestDB1):
CREATE TABLE dbo.Stuff
(
[Id] INT IDENTITY(1, 1) NOT NULL PRIMARY KEY,
[Something] NVARCHAR(50) NOT NULL
);
Audit Table (in TestDB2):
CREATE TABLE dbo.AuditLog
(
AuditLogID INT IDENTITY(1, 1) NOT NULL PRIMARY KEY,
EventTime DATETIME NOT NULL DEFAULT (GETDATE()),
BeforeValue NVARCHAR(50) NULL,
AfterValue NVARCHAR(50) NULL
);
SQLCLR Trigger on main table (partial code):
string _AuditSQL = #"
INSERT INTO TestDB2.dbo.AuditLog (BeforeValue, AfterValue)
SELECT del.Something, ins.Something
FROM INSERTED ins
FULL OUTER JOIN DELETED del
ON del.Id = ins.Id;
";
SqlConnection _Connection = new SqlConnection("Context Connection = true");
SqlCommand _Command = _Connection.CreateCommand();
_Command.CommandText = _AuditSQL;
try
{
_Connection.Open();
_Command.ExecuteNonQuery();
}
finally
{
_Command.Dispose();
_Connection.Dispose();
}
Test queries:
USE [TestDB1];
SELECT * FROM dbo.Stuff;
---
INSERT INTO dbo.Stuff (Something) VALUES ('qwerty');
INSERT INTO dbo.Stuff (Something) VALUES ('asdf');
SELECT * FROM dbo.Stuff;
SELECT * FROM TestDB2.dbo.AuditLog;
---
UPDATE tab
SET tab.Something = 'dfgdfgdfgdfgdfgdfgd'
FROM dbo.Stuff tab
WHERE tab.Id = 2;
SELECT * FROM dbo.Stuff;
SELECT * FROM TestDB2.dbo.AuditLog;
It looks like it's not possible. However a T-SQL statement can reference another database in the same instance by using [DatabaseName].[dbo].[TableName]. I can do the messy logic in my CLR trigger, then do a final insert into the second database by calling a simple T-SQL stored procedure and passing in parameters.
after connecting to database in C#
string MyConString2 = "SERVER=localhost;" + "user id=mytest;" + "DATABASE=clusters;" + "PASSWORD=mypass;";
I have an algorithm which I need to after each run of algorithm that will fill the database, drop the database "clusters" of mysql manually and again connect to the empty database and run it again,gaining new data in tables
I want to make it automatically how can I drop or empty my database if exists in C# and then run my algorithm?
Here is example code that works and I think this is what you are talking about, if not, feel free to correct me.
using (var connection = new MySqlConnection("server=localhost;user=root;password="))
{
connection.Open();
var command = connection.CreateCommand();
command.CommandText = "drop schema if exists clusters";
command.ExecuteNonQuery();
command = connection.CreateCommand();
command.CommandText = "create schema clusters";
command.ExecuteNonQuery();
}
Prepare sql query for clearing your DB and test it in f.e. MySQL workbench. Following this, just execute it as you would execute regular query against DB in C#. One way is to clear all the tables in your database by using TRUNCATE statement and the second way is to DROP DATABASE and recreate it.