Is there a way to call a Snowflake Stored Procedure asynchronously from .Net? I am running a .Net Core API App using Snowflake’s DotNet driver. Basically I need to be able to use conn.OpenAsync and cmd.ExecuteNonQueryAsync like you normally would with a SQL Server, but that seems to be impossible.
executeResults.SessionId =RunNonQueryAsync(connectionString, command).Result;
static async Task<String> RunNonQueryAsync(string execConnection, string execCommand)
{
String sessionId = null;
using (IDbConnection conn = new SnowflakeDbConnection())
{
conn.ConnectionString = execConnection;
conn.Open();
// Get Session Information
IDbCommand cmd2 = conn.CreateCommand();
cmd2.CommandText = "SELECT CURRENT_SESSION() as SESSION_ID";
IDataReader rdr2 = cmd2.ExecuteReader();
while (rdr2.Read())
{
sessionId = rdr2.GetValue(0).ToString();
}
Task taskA = new Task(() =>
{
using (IDbConnection cn = new SnowflakeDbConnection())
{
cn.ConnectionString = execConnection;
cn.Open();
IDbCommand cm = conn.CreateCommand();
cm.CommandText = "call StoredProcedureThatTakes5MinutesToRun";
cn.Close();
}
});
taskA.Start();
}
return sessionId;
}
My browser sends an HTTP GET which the DotNet API receives along with the properties [SQL Type: "NonQuery" AND StatementText:"Call StoredProcX"].
That sounds like a absolute security nightmare. SQL Injections are bad enough, but that is basically putting a Central Venous Catheter into your Database.
At best, I would allow some simple Enumeration, integer or string value to be send with the request. One that is then passed through a switch/case statement. Maybe fed into a Dictionary<String, Action>, if that works better. The case or the action will then do the work of sending off the requests.
From there it is really just doing two things within then same connection:
using (IDbConnection cn = new SnowflakeDbConnection())
{
cn.Open();
//Build command 1
//Execute command 1
//Do stuff with results of command 1
//Build command 2
//Execute command 2
//Do stuff with results of command 2
//Can propably be omited, as Dispose usually includes close. But we are talking about some 3rd party code, that might not be tested that well
cn.Close();
}
Related
I have created two different connections. One for DataReader and second for NonQuery.
DataReaderConnect = DBUtils.GetDBConnection(_DbUserName, _DbPassword);
DataReaderConnect.Open();
NonQueryConnect = DBUtils.GetDBConnection(_DbUserName, _DbPassword);
NonQueryConnect.Open();
I have 2 methods that is using those connections
public async Task UpdateUser(cUser.UserObject User)
{
string SQL = $"UPDATE {_DbName}.dbUserTable SET Name = #Name....";
using MySqlCommand Command = new MySqlCommand(SQL, NonQueryConnect);
Command.Parameters.AddWithValue("#Name", User.Name);
...
await Command.ExecuteNonQueryAsync().ConfigureAwait(false);
}
public async Task<cUser.UserObject> GetUser(int ttvUserID)
{
List<cUser.UserObject> Users = new List<cUser.UserObject>();
string SQL = $"SELECT * FROM {_DbName}.dbUserTable WHERE TwitchID = #ID";
using MySqlCommand Command = new MySqlCommand(SQL, DataReaderConnect);
Command.Parameters.AddWithValue("#ID", ttvUserID);
using var sqlReader = await Command.ExecuteReaderAsync().ConfigureAwait(false);
while (await sqlReader.ReadAsync().ConfigureAwait(false))
{
Users.Add(new cUser.UserObject()
{
dbID = Convert.ToInt32(sqlReader[0]),
TwitchID = Convert.ToInt32(sqlReader[1]),
Name = sqlReader[2].ToString(),
isSub = Convert.ToInt32(sqlReader[3]),
isVip = Convert.ToInt32(sqlReader[4])
});
}
}
But im getting exception "There is already an open DataReader associated with this Connection which must be closed first." And I cant figure out why. Im not using DataReader for NonQuery and vice versa
edit: I figured it out and it was just me being silly and apparently I can't read. .NET not only cant support DataReader and NonQuery on the same connection but also more then one DataReader on the same connection.
Don't keep multiple open connections to the database. In fact, don't keep open connections at all to the database. It leaves hanging resources that the underlying system is better equipped to manage than our code is.
These interactions with the database should:
Open the connection
Use the connection
Close the connection
So instead of this:
public async Task UpdateUser(cUser.UserObject User)
{
string SQL = $"UPDATE {_DbName}.dbUserTable SET Name = #Name....";
using MySqlCommand Command = new MySqlCommand(SQL, NonQueryConnect);
Command.Parameters.AddWithValue("#Name", User.Name);
//...
await Command.ExecuteNonQueryAsync().ConfigureAwait(false);
}
Instead, use something like this (other issues notwithstanding):
public async Task UpdateUser(cUser.UserObject User)
{
using var connection = DBUtils.GetDBConnection(_DbUserName, _DbPassword);
var sql = $"UPDATE {_DbName}.dbUserTable SET Name = #Name....";
using var command = new MySqlCommand(sql, connection);
command.Parameters.AddWithValue("#Name", User.Name);
//...
await command.ExecuteNonQueryAsync().ConfigureAwait(false);
}
This keeps the scope of the connection itself limited to just this operation, since both command and connection will be disposed at the end of their respective using blocks. That way you don't have open connections hanging around that aren't being used, and you don't run into errors like the one you're seeing now.
I have the following INSERT method in a C# web project. If I run the project without MySQL connection poling everything works fine, but when I activate Pooling=True in the DB connection string this method stops working, the insert statements never complete.
I realized how to modify the code to make it work, but I would like to understand what is happening and I hope you could help.
When I comment line //myR.Close(); everything works fine.
using MySql.Data.MySqlClient;
//query example consulta="INSERT INTO users (id, name) VALUES (1, 'Rob');
public static MySqlConnection GetWriteConnection()
{
string connStr = MySqlConnectionStrings.WriteConnectionString;
MySqlConnection conn = new MySqlConnection(connStr);
return conn;
}
public static MySqlConnection GetReadConnection()
{
string connStr = MySqlConnectionStrings.ReadConnectionString;
MySqlConnection conn = new MySqlConnection(connStr);
return conn;
}
public static bool Insert(string consulta)
{
MySqlConnection conn = BdaHelper.GetWriteConnection();
conn.Open();
using (conn)
{
try
{
if (conn.State == ConnectionState.Closed)
{
conn.Open();
}
MySqlCommand micomando = new MySqlCommand(consulta, conn);
micomando.ExecuteNonQuery(); //still not working
return true;
}
catch (Exception ex)
{
return false;
}
}
}
My app has also multi-thread concurrency and two types of database connections, one specifically for only-read purposes and other different for write. When an insert statement fails I don't get any error simply the change doesn't commit in the database. Reading the article in the comments I don't think this applies to this issue but I would add an example of my main program:
MySqlConnection readConnection = BdaHelper.GetReadConnection();
using (readConnection)
{
var users = GetUsers(readConnection);
var credentials = GetCredentials(readConnection);
//Example is the query that fails don't giving any exception
Insert("INSERT INTO login_log (id_user, date) VALUES (1, now())");
}
May the problem be caused because there are two concurrent connections?
I shouldn't reuse read connection, even is a different connection than the write connection?
When you search the internet or SO how to connect to SQL database inside Script Task in SSIS you will find .NET v1.1ish code like this:
ConnectionManager cm;
System.Data.SqlClient.SqlConnection sqlConn;
System.Data.SqlClient.SqlCommand sqlComm;
cm = Dts.Connections["ADO.NET.SqlDB"];
sqlConn = (System.Data.SqlClient.SqlConnection)cm.AcquireConnection(Dts.Transaction);
sqlComm = new System.Data.SqlClient.SqlCommand("your SQL Command", sqlConn);
sqlComm.ExecuteNonQuery();
cm.ReleaseConnection(sqlConn);
I am looking for updated code that makes good use of later introduced .NET features.
For a start, how about the code below. Is this the current recommended way to connect to SQL Server inside Script Task in SSIS 2012 and later or do I miss something here?
ConnectionManager cm = Dts.Connections["ADO.NET.SqlDB"];
using (var sqlConn = (SqlConnection)cm.AcquireConnection(Dts.Transaction))
{
if (sqlConn.State != ConnectionState.Open)
sqlConn.Open();
using (var sqlComm = new SqlCommand(
String.Format("UPDATE myTab SET Processed = 4 where ID = '{0}'",
idNumber), sqlConn))
{
return sqlComm.ExecuteNonQuery();
}
}
Is the ReleaseConnection() still needed?
Is sqlConn.Open() really needed in an SSIS context?
One year later, and hopefully a little wiser, I settled with code like this:
ConnectionManager cm = Dts.Connections["ADO.NET.SqlServerDB"];
var sqlConn = (SqlConnection)cm.AcquireConnection(Dts.Transaction);
using (var sqlCmd = new SqlCommand(
"INSERT INTO apou_moodle_import(id_num, username, email) VALUES(#IdNum, #Username, #Email)",
sqlConn))
{
sqlCmd.CommandType = CommandType.Text;
sqlCmd.Parameters.AddWithValue("#IdNum", newUser.id_num);
sqlCmd.Parameters.AddWithValue("#Username", newUser.username);
sqlCmd.Parameters.AddWithValue("#Email", newUser.email);
int rowsAffected = sqlCmd.ExecuteNonQuery();
}
cm.ReleaseConnection(sqlConn);
So, I keep using ConnectionManager.ReleaseConnection, however, SqlConnection.Open & Close are not needed in an SSIS context. Plus, use Parameters to play safe.
Well, using structure allows you to automate disposing variables and handle it better. However, sqlConn is not a simple class, it is a ConnectionManager instance. When you start using it, you call AcquireConnection, when end - call ReleaseConnection. The ReleaseConnection may perform some housekeeping specific to this Data Source. Depending on ConnectionManager implementation, it may check at disposal whether ReleaseConnection was called or not and call it.
To my understanding, your code with using should be Ok in most cases. Problems may arise when you repeatedly open connections and do not release it - you might run of connection pool etc. I would wrap internals into try - finally block to ensure ReleaseConnection execution.
I have a .NET web service application that executes parameterized MS SQL Server stored procedures using System.Data.SqlCommand. The application receives a name of the stored procedure and its parameters from user input.
Also the application deployed within Windows AD domain and it is able to obtain the name of a remote user with the help of SSPI authentication.
using (var con = new SqlConnection(connectionString)) {
using (var cmd = new SqlCommand(procedureName, con)) {
cmd.CommandType = CommandType.StoredProcedure;
foreach (var pair in pairs.AllKeys) {
cmd.Parameters.Add(new SqlParameter(pair, pairs[pair]));
}
con.Open();
using (var reader = cmd.ExecuteReader()) {
// processing results
}
}
}
Now I want to execute a stored procedure with an EXECUTE AS statement.
use [db]
go
execute as user = 'domain\user' --execute as RemoteUser
exec [db].[stored_procdure] #param1=value1
Can this be done? How can I add EXECUTE AS to the SqlCommand?
I would not like to resort to sql injection prone code and build the sql request from strings received from user.
Solved it some time ago with a colleague of mine.
To achieve the requested behavior the execute as statement should be run in a separate preceeding SqlCommand while in the same SqlConnection.
After the closing of the reader, still in the same SqlConnection, there's another separate SqlCommand needed - revert - to switch back the context.
I was having a similar issue where I was able to Execute as User but the Revert wasn't working. By following prot's answer I was able to fix it. So for anyone having a similar issue this is how the code looks
SqlCommand cmd = new SqlCommand("EXECUTE AS USER = 'domain\\user';");
OSSDBDataContext dc = new OSSDBDataContext();
cmd.Connection = dc.Connection as SqlConnection;
cmd.Connection.Open();
cmd.ExecuteNonQuery();
//Execute stored procedure code goes here
SqlCommand cmd2 = new SqlCommand("REVERT;");
cmd2.Connection = dc.Connection as SqlConnection;
cmd2.ExecuteNonQuery();
How to write tests for delegate methods?
or
Beware of 2 open connections both with 'hooks' onto the same SQL table .... .
This was not straight forward to diagnose, test and prove is not a problem with my current solution.
How could I have TDD'd or written unit/integration tests to have trapped this? Redesign suggestions ...
Create a connection to the table 'TransferWebTransmit' to process
all rows.
Execute a Reader to loop through 'old' records, (ID=1)
Call a delegate method to process the 'old' record. (NB keep current
connection open until all rows are processed i.e. have called the delegate).
Delegate method:
Opens a new connection, executes a Stored Proc 'TransferWebTransmitUpdate'
which -> Updates the table 'TransferWebTransmit' row (ID=1), then does a SELECT on (ID=1) row
----> cursor lock!
----> .Net throws "System.Data.SqlClient.SqlException (0x801
31904): Timeout expired. The timeout period elapsed prior to completion of the
operation or the server is not responding".
----> Connections are locked.
----> Have to Kill processes to recover
Here's the delegate method:
public int Update(int transferID)
{
var obj = new TransferWebMessage();
using (SqlConnection conn = base.GetNewConnection())
{
using (SqlCommand sp_cmd = new SqlCommand())
{
sp_cmd.CommandText = "TransferWebTransmitUpdate";
sp_cmd.CommandType = CommandType.StoredProcedure;
sp_cmd.Parameters.AddWithValue("TransferID", transferID);
sp_cmd.Connection = conn;
conn.Open();
SqlDataReader rdr = sp_cmd.ExecuteReader();
int roweffected;
while (rdr.Read())
{
roweffected = rdr.GetInt32(0),
}
}
}
return roweffected;
}
Here's the call to get the rows to process and call the delegate:
public void WatchForDataTransferRequests(_delegateMethod callback)
{
using (SqlConnection conn = new SqlConnection(_insol_SubscriberConnectionString))
{
// Construct the command to get any new ProductReview rows from the database along with the corresponding product name
// from the Product table.
SqlCommand cmd = new SqlCommand(
"SELECT [TransferID]" +
" FROM [dbo].[TransferWebTransmit]" +
" ORDER BY [TransferID] ASC", conn);
cmd.Parameters.Add("#CurrentTransferID", SqlDbType.Int);
conn.Open();
SqlDataReader rdr = cmd.ExecuteReader();
// Process the rows
while (rdr.Read())
{
Int32 transferID = (Int32)rdr.GetInt32(0);
callback(transferID);
}
}
}