Asked this question a few days ago but maybe wasn't too specific.
Basically, I'm writing a console app that accepts a list of stored procedure names and arbitrarily executes them. The app is supposed to log to Slack, etc. on the progress of these sprocs, as well as do some other stuff later.
The 'other stuff' I mentioned interacts with known database models, and since we have a library of ServiceStack/OrmLite types build up, I am using OrmLite to interact with the database later.
So, since I have OrmLite included and configured in the project, I'm just using the OrmLite IDbConnection to execute the stored procedures, even if it's just by name. The sprocs simply move data to another database, one that the app doesn't need to interact with -- so I'm not SELECTing any data into the app with the sprocs and there's no POCO representation of what's moved.
However, the sprocs spit out a number of informational PRINT statements that I want to capture and log to our team in a formatted manner via Slack. Enabling full-on debug logging spits out a little too much information for this purpose.
Is there any way I can configure the Ormlite SqlServerDialect.Provider, the IDataReader, etc. to basically capture the SqlConnection.InfoMessage
event (or to provide it with a connection already prepped with a SqlInfoMessageEventHandler)?
The logging strategy approach for doing this is to configure a filter in your logging provider to only log the messages you're interested in.
OrmLite's SQL debug logging messages containing the SQL and DB Params are logged under the ServiceStack.OrmLite.OrmLiteResultsFilterExtensions Type.
To make this scenario easier I've just added new OrmLiteConfig.BeforeExecFilter and OrmLiteConfig.AfterExecFilter which you can use to execute custom logic before and after a DB Command is executed, e.g:
OrmLiteConfig.BeforeExecFilter = dbCmd => Console.WriteLine(dbCmd.GetDebugString());
This change is available from v5.0.3 that's now available on MyGet.
Update:
Looking through the ServiceStack.OrmLite source, it looks like IDbConnection can be cast to SqlConnection using (SqlConnection)IDbConnection.ToDbConnetion().
Wrapping this up in an OrmLiteExecFilter to be called on every statement, casting the IDbConnection and creating the DbCommand with the cast connection worked for me:
public class LogInfoMessageFilter : OrmLiteExecFilter
{
ILog SlackLog = LogManager.GetLogger("SlackLogger");
public override T Exec<T>(IDbConnection dbConn, Func<IDbCommand, T> filter)
{
var holdProvider = OrmLiteConfig.DialectProvider;
// casting, skipping type checks for brevity
var sqlConn = (SqlConnection)dbConn.ToDbConnection();
// add the event
sqlConn.InfoMessage += _HandleInfoMessage;
var dbCmd = CreateCommand(sqlConn);
try
{
return filter(dbCmd);
}
finally
{
DisposeCommand(dbCmd, sqlConn);
OrmLiteConfig.DialectProvider = holdProvider;
}
}
private void _HandleInfoMessage(object sender, SqlInfoMessageEventArgs args)
{
SlackLog.Info($"what does the sproc say? {args.Message}");
}
}
// before opening the connection:
OrmLiteConfig.ExecFilter = new LogInfoMessageFilter();
However, now that #mythz has replied with an 'official' way to do this, I'll go about refactoring. Just thought I'd plop this here in case it suits anyone's use case.
Related
I am trying to setup my .NET 4.7.1 program that is connecting to a MySQL database 8.0 to use the minimum privileges to run.
The .NET program is using MySql.Data to make connection. The minimum right for a user to execute a stored procedure is typically only EXECUTE privilege. This works fine from MySQL workbench or command line.
Upon running the .NET program this does return the following exception:
System.Data.SqlTypes.SqlNullValueException: 'Data is Null. This method or property cannot be called on Null values.'
To make it easy, I have create a very small demo program to demonstrate the issue.
Setup of the database:
CREATE DATABASE Spike;
CREATE PROCEDURE TestAccess()
BEGIN
END;
CREATE USER Spike#localhost IDENTIFIED WITH mysql_native_password BY 'sample';
GRANT EXECUTE ON PROCEDURE `TestAccess` TO Spike#localhost;
Setup program code:
static void Main(string[] args)
{
using (MySqlConnection conn = new MySqlConnection("Server=localhost;Database=Spike;uid=Spike;pwd=sample"))
{
conn.Open();
Console.WriteLine("Connection open");
MySqlCommand cmd = new MySqlCommand();
cmd.Connection = conn;
cmd.CommandText = "TestAccess";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.ExecuteNonQuery();
Console.WriteLine("Query executed");
}
Console.ReadKey();
}
The crash happens at the line cmd.ExecuteNonQuery();
The stack from the crash is interesting, since it seems to indicate that the information_schema is queried. When logging all statements I can see that the last statement before the exception is:
SELECT * FROM information_schema.routines WHERE 1=1 AND routine_schema LIKE 'Spike' AND routine_name LIKE 'TestAccess'
I cannot grant different rights on information_schema, but I could give more rights on the stored procedure to make more information visible in the routines table, this feels wrong however. Simple tests with granting CREATE and ALTER access also did not work.
Is there something else I can do, without granting too much privileges?
This appears to be a bug in Connector/NET, similar to bug 75301 but a little different. When it's trying to determine parameter metadata for the procedure, it first creates a MySqlSchemaCollection named Procedures with all metadata about the procedure. (This is the SELECT * FROM information_schema.routines WHERE 1=1 AND routine_schema LIKE 'Spike' AND routine_name LIKE 'TestAccess' query you see in your log.)
The Spike user account doesn't have permission to read the ROUTINE_DEFINITION column, so it is NULL. Connector/NET expects this field to be non-NULL and throws a SqlNullValueException exception trying to read it.
There are two workarounds:
1) The first, which you've discovered, is to set CheckParameters=False in your connection string. This will disable retrieval of stored procedure metadata (avoiding the crash), but may lead to harder-to-debug problems calling other stored procedures if you don't get the order and type of parameters exactly right. (Connector/NET can no longer map them for you using the metadata.)
2) Switch to a different ADO.NET MySQL library that doesn't have this bug: MySqlConnector on NuGet. It's highly compatible with Connector/NET, performs faster, and fixes a lot of known issues.
I found an answer with which I am quite pleased. It is changing the connection string by adding CheckParameters=false:
using (MySqlConnection conn = new MySqlConnection("Server=localhost;Database=Spike;uid=Spike;pwd=sample;CheckParameters=false"))
This disables parameter checking, and thereby information_schema queries.
I am trying to use sql Dependency to check that a table data has been changed by other clients I followed the turotial here but when I try to create the queue i get the following error message this is my c# code and sql statement.
Edit 1
Sorry I forgot to include the tutorial in my first post here is the link in the tutorial it states quename but does not say how to ccreate it.
https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/detecting-changes-with-sqldependency
CREATE QUEUE ExportedOrdersChangeMessage CREATE SERVICE
ExportedOrdersService ON QUEUE ExportedOrdersChangeMessage
([http://schemas.microsoft.com/SQL/Notification/PostQueryNotification])
The error I receive after running the above is the following.
Msg 15151, Level 16, State 1, Line 5
c'http://schemas.microsoft.com/SQL/Notification/PostQueryNotification',
because it does not exist or you do not have permission.
Basically what I want to be able to do is if a user is atempting to change a record I want to send a notification to the other clients. Is this the best method for this approach its a winforms business app if any other libarys that make this easier would be great.
I am using sql Server 2017 express which I asumes would have this as standard?.
private void DetectChanges()
{
// Assume connection is an open SqlConnection.
string connectionString = ConfigurationManager.AppSettings["ConnectionString"];
using (SqlConnection con = new SqlConnection(connectionString))
{
// Create a new SqlCommand object.
using (SqlCommand command = new SqlCommand("SELECT exportedtoMeteor FROM dbo.exported", con))
{
// Create a dependency and associate it with the SqlCommand.
SqlDependency dependency = new SqlDependency(command);
// Maintain the reference in a class member.
SqlDependency.Start(connectionString, "ExportedOrdersChangeMessage");
// Subscribe to the SqlDependency event.
dependency.OnChange += new
OnChangeEventHandler(OnDependencyChange);
// Execute the command.
using (SqlDataReader reader = command.ExecuteReader())
{
// Process the DataReader.
}
}
}
}
// Handler method
void OnDependencyChange(object sender, SqlNotificationEventArgs e)
{
// Handle the event (for example, invalidate this cache entry).
}
void Termination()
{
string connectionString = ConfigurationManager.AppSettings["ConnectionString"];
// Release the dependency.
SqlDependency.Stop(connectionString, ExportedHacket);
}
I don't know the answer to your direct question but Butterfly Server .NET detects changes to databases (MySQL, Postgres, SQLite, MS SQL Server, etc) and allows creating Dynamic Views based on SELECT statements that publish events if the SELECT results change. There are caveats (like all database operations have to be performed via the IDatabase interface provided by Butterfly Server .NET). However, it's similar enough to what you are doing that you may be interested.
Here I need to get a string that identifies the database client using PostgreSQL with C#. Here is the following example.
Example:
private static void OpenNpgsqlConnection(string connectionString)
{
using (NpgsqlConnection connection = new NpgsqlConnection(connectionString))
{
connection.Open();
Console.WriteLine("WorkstationId: {0}", connection.WorkstationId);
}
}
Error:
'Npgsql.NpgsqlConnection' does not contain a definition for 'Workstationid'.
You seem to mean the MS SQL specific property WorkstationId of SQLConnection.
This is an MS SQL specific property. It does not exist for nPgSQL, though of course you could add it.
The documentation in MSDN is profoundly useless, so it's hard to tell what it's actually for. At a wild guess, I think you probably mean something like PostgreSQL's application_name - a way for the server to find out information about the client from within a stored procedure or function.
client drivers or applications have to set application_name; there's nothing that gets set automatically to the "network name". (What would that even mean if the client was not a Windows box?). Your application could set application_name, either in the connection string or via an explicit SET command.
You can also run arbitrary SET commands. E.g you might decide to store the "workstation name" in the myapp.workstationid connection property. So you:
SET myapp.workstationid = 'BOBSCOMPUTER';
from your app, after making the initial connection.
Stored procedures can now access it with current_setting('myapp.workstationid').
Note that in both cases, a user with the ability to send raw SQL can just replace these settings with whatever they want. So don't use them for security.
I'm trying to write some unit tests for my code that connect to SQL Server for persistence, but I would like to be able to run my unit tests by just pointing it at a SQL Server instance, and let the tests create their own database to run tests in, so after each test it can just drop the database and then on setup before the next test recreate it, so I know there is no legacy data or structures left over from a previous test effecting the next test.
In brief: no, you cannot do that. You might be able to leave out the database from the connection string, but in that case, that connection will be made to the configured default database of the login that's connecting to SQL Server (and that default database must exist at the time the connection is made)
If you want to have this scenario, you need to
first connect to your instance and database master and create your new testdb (or whatever it's called)
disconnect
in your tests, connect to the instance and the testdb database
Better yet: use a mocking framework of some sort so you don't even need an actual database in your testing scenario!
I use the following class to facilitate the OP's scenario:
public class MsSqlDatabaseCreator
{
public void Create(string connectionstring)
{
if (DatabaseExists(connectionstring))
{
DropDatabase(connectionstring);
}
CreateDatabase(connectionstring);
}
private static void CreateDatabase(string connectionString)
{
var sqlConnectionStringBuilder = new SqlConnectionStringBuilder(connectionString);
var databaseName = sqlConnectionStringBuilder.InitialCatalog;
sqlConnectionStringBuilder.InitialCatalog = "master";
using (var sqlConnection = new SqlConnection(sqlConnectionStringBuilder.ConnectionString))
{
sqlConnection.Open();
using (var sqlCommand = sqlConnection.CreateCommand())
{
sqlCommand.CommandText = $"CREATE DATABASE {databaseName}";
sqlCommand.ExecuteNonQuery();
}
}
}
private static bool DatabaseExists(string connectionString)
{
var sqlConnectionStringBuilder = new SqlConnectionStringBuilder(connectionString);
var databaseName = sqlConnectionStringBuilder.InitialCatalog;
sqlConnectionStringBuilder.InitialCatalog = "master";
using (var sqlConnection = new SqlConnection(sqlConnectionStringBuilder.ConnectionString))
{
sqlConnection.Open();
using (var command = sqlConnection.CreateCommand())
{
command.CommandText = $"SELECT db_id('{databaseName}')";
return command.ExecuteScalar() != DBNull.Value;
}
}
}
private static void DropDatabase(string connectionString)
{
var sqlConnectionStringBuilder = new SqlConnectionStringBuilder(connectionString);
var databaseName = sqlConnectionStringBuilder.InitialCatalog;
sqlConnectionStringBuilder.InitialCatalog = "master";
using (var sqlConnection = new SqlConnection(sqlConnectionStringBuilder.ConnectionString))
{
sqlConnection.Open();
using (var sqlCommand = sqlConnection.CreateCommand())
{
sqlCommand.CommandText = $#"
ALTER DATABASE {databaseName} SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
DROP DATABASE [{databaseName}]
";
sqlCommand.ExecuteNonQuery();
}
}
}
}
The important part is the switching of the database name (initial catalog) to master. This way you can have just one connectionstring.
What you want to accomplish is possible using a mocking framework, in which case you don't even have to "connect to a database", you simply mock the return values that the database should return in order for you to test your "db handler" implementation.
There are several to choose from when it comes to C#, I can recommend Rhino Mocks and Moq to name two. Here's a question detailing a bit more; https://stackoverflow.com/questions/37359/what-c-sharp-mocking-framework-to-use
Why not have the same named database dedicated for tests? and drop-create it every time. This way you won't need to mess about with connection strings - it is always the same.
And yet, there is a better solution: within all your tests, start transaction, do your test, where your data is messed up. Once you verified (or failed) the test, unroll the transaction. This way you don't need to drop-create your tests for every test, because the data is never changed.
But you'll need to make sure schema in test-database is always up to date. So you'll need to drop-create test database whenever your schema is changed.
I've blogged about database tests and how we deal with Entity Framework migrations. This might not be completely applicable to your situation, but might help with ideas.
Regarding using mocks in your tests - yes this is absolutely valid suggestion and should be followed most of the time. Unless you are trying to test the database layer. In that case no mocks will save you, and you just have to go to DB. Many times over I have tried to mock DbContext in EF, but never managed to simulate realistic DB behavior. So going to DB was easier for me, rather than simulating DB-mock.
I'd use SQL Server Management Objects for the task. It's Server and Database APIs doesn't necessarily need a connection string but I think you might still need to specify a database. You can use master for that. (Check jeroenh's answer about creating object using SMO API as well)
By the way, if you are using .Net 4.0.2 and up you can use LocalDB as well, which is even better.
Edit: Note that actually LocalDB is an SQL Server 2012 feature however you still need .Net Framework > 4.0.2 to be able to use it.
I am using the devart component dotconnect for postgresql. I have created the site using linq to entities, however, I would like each user to have a seperate database. This means that I need to change the connection string for each person that has logged in. I understand the main part of how to generate a new connection string etc, however, when i pass that as a paramater to the object context object it comes back with the error
"user id keyword not supported, "
if i create a class that generates an entity connection the error message changes to:
"Unable to load the specified metadata resource."
Cannot work out what I have done wrong in these instances.
ok so, as usual, when i posted this question, about 3 minutes later i found the problem. The entity connection string, for general purposes should have a cool little
res://*/
this makes the metadata work. This solves the problem of metadata resource and this works. So to help others who may, like me, have spent development time doing this, i created a class, with a method like so.
public static string getConnString(string database)
{
string connectionstring = "User Id=USER ID HERE;Password=PASSWORD HERE;Host=server;Database="+database+";Persist Security Info=True;Schema=public";
EntityConnectionStringBuilder newconnstring = new EntityConnectionStringBuilder();
newconnstring.Metadata = #"res://*/";
newconnstring.Provider = "Devart.Data.PostgreSql";
newconnstring.ProviderConnectionString = connectionstring;
return newconnstring.ToString();
}
then create a constructor like so
dataEntities data = new dataEntities(databaseConnection.getConnString(INSERTDBNAMEHERE);
Then we can reference that in the same way as a usual linq statement. Simples!!