I am having an issue with my code and the numerous SQL calls that it makes with deadlocks. I pasted the code into PasteBin here: https://pastebin.com/p1YDkKsB. Can someone help me out here? It happens most often in the CheckClanActivity task here:
using(SqlCommand cmd = new SqlCommand(string.Format("select * from ClanMembers where MembershipId={0} and IsActive = 1", entry.Player.DestinyUserInfo.MembershipId), conn))
, but it also happens all over the place.
Edit:
Okay, the involved SQL statements are as follows:
if not exists(select * from ClanMembers where MembershipId={0}) begin insert into ClanMembers(ID, MembershipId, BattleNetId, ClanId, DateLastPlayed, IsActive, LastUpdated) select ISNULL(MAX(ID) + 1, 0),{0},'{1}',{2},'{3}', 1, GETDATE() from ClanMembers end else begin update ClanMembers set DateLastPlayed='{3}', LastUpdated=GETDATE() where MembershipId={0} end
if not exists(select * from ClanMemberCharacters where CharacterId={1}) begin insert into ClanMemberCharacters(ID,MembershipId,CharacterId) select ISNULL(MAX(ID)+1,0),{0},{1} from ClanMemberCharacters end
select c.* from ClanMemberCharacters c join ClanMembers m on m.MembershipId = c.MembershipId where m.IsActive = 1 ORDER BY m.MembershipId desc
select * from ActivityHistory where InstanceId = {0}
if not exists(select * from ActivityHistory where InstanceId = {0}) begin insert into ActivityHistory(InstanceId,MembershipId,CharacterId,GameMode,ActivityDate,ReferenceId,DirectorActivityHash,IsPrivate,ClanActivity,ClanActivityCount) values( {0},{1},{2},'{3}','{4}',{5},{6},{7},{8},{9} ) END
select * from ClanMembers where IsActive = 1
Select * from ClanMembers where IsActive = 0
update ClanMembers set IsActive = 1 where MembershipId = {0}
select * from ClanMembers where MembershipId={0} and IsActive = 1
I suspect the problem is the nesting of SqlConnections with the same connection string one within the other. Suggestions:
Move the connection using as close to the scope of the command as possible. For example, avoid calls to your other methods from within it.
Avoid the if (conn.State == ConnectionState.Closed) { conn.Open(); } and simply move the conn.Open(); to be the first thing inside the using of the SqlConnection.
Remove the conn.Close() which will be done automatically when leaving the using block.
SqlDataAdapter is also disposable so should be in a using block.
The way you catch and swallow exceptions means the caller has no idea that something has gone wrong. Consider rethrowing the exception with throw; or not catching the exception here, and catching it higher up the call stack.
The way you are logging the errors will not help you diagnose the problem. Consider logging ex.ToString() rather than just ex.Message.
Separate your concerns. You are cramming everything together. Refactor your code so that the data layer is separated from the business logic.
Avoid doing string.Format to construct your queries, because by doing so they are vulnerable to SQL injection attacks.
Related
I'm writing a C# class library in which one of the features is the ability to create an empty data table that matches the schema of any existing table.
For example, this:
private DataTable RetrieveEmptyDataTable(string tableName)
{
var table = new DataTable() { TableName = tableName };
using var command = new SqlCommand($"SELECT TOP 0 * FROM {tableName}", _connection);
using SqlDataAdapter dataAdapter = new SqlDataAdapter(command);
dataAdapter.Fill(table);
return table;
}
The above code works, but it has a glaring security vulnerability: SQL injection.
My first instinct is to parameterize the query like so:
using var command = new SqlCommand("SELECT TOP 0 * FROM #tableName", _connection);
command.Parameters.AddWithValue("#tableName", tableName);
But this leads to the following exception:
Must declare the table variable "#tableName"
After a quick search on Stack Overflow I found this question, which recommends using my first approach (the one with sqli vulnerability). That doesn't help at all, so I kept searching and found this question, which says that the only secure solution would be to hard-code the possible tables. Again, this doesn't work for my class library which needs to work for arbitrary table names.
My question is this: how can I parameterize the table name without vulnerability to SQL injection?
An arbitrary table name still has to exist, so you can check first that it does:
IF EXISTS (SELECT 1 FROM sys.objects WHERE name = #TableName)
BEGIN
... do your thing ...
END
And further, if the list of tables you want to allow the user to select from is known and finite, or matches a specific naming convention (like dbo.Sales%), or belongs to a specific schema (like Reporting), you could add additional predicates to check for those.
This requires you to pass the table name in as a proper parameter, not concatenate or token-replace. (And please don't use AddWithValue() for anything, ever.)
Once your check that the object is real and valid has passed, then you will still have to build your SQL query dynamically, because you still won't be able to parameterize the table name. You still should apply QUOTENAME(), though, as I explain in these posts:
Protecting Yourself from SQL Injection in SQL Server - Part 1
Protecting Yourself from SQL Injection in SQL Server - Part 2
So the final code would be something like:
CREATE PROCEDURE dbo.SelectFromAnywhere
#TableName sysname
AS
BEGIN
IF EXISTS (SELECT 1 FROM sys.objects
WHERE name = #TableName)
BEGIN
DECLARE #sql nvarchar(max) = N'SELECT *
FROM ' + QUOTENAME(#TableName) + N';';
EXEC sys.sp_executesql #sql;
END
ELSE
BEGIN
PRINT 'Nice try, robot.';
END
END
GO
If you also want it to be in some defined list you can add
AND #TableName IN (N't1', N't2', …)
Or LIKE <some pattern> or join to sys.schemas or what have you.
Provided nobody has the rights to then modify the procedure to change the checks, there is no value you can pass to #TableName that will allow you to do anything malicious, other than maybe selecting from another table you didn’t expect because someone with too much access was able to create before calling the code. Replacing characters like -- or ; does not make this any safer.
You could pass the table name to the SQL Server to apply quotename() on it to properly quote it and subsequently only use the quoted name.
Something along the lines of:
...
string quotedTableName = null;
using (SqlCommand command = new SqlCommand("SELECT quotename(#tablename);", connection))
{
SqlParameter parameter = command.Parameters.Add("#tablename", System.Data.SqlDbType.NVarChar, 128 /* nvarchar(128) is (currently) equivalent to sysname which doesn't seem to exist in SqlDbType */);
parameter.Value = tableName;
object buff = command.ExecuteScalar();
if (buff != DBNull.Value
&& buff != null /* theoretically not possible since a FROM-less SELECT always returns a row */)
{
quotedTableName = buff.ToString();
}
}
if (quotedTableName != null)
{
using (SqlCommand command = new SqlCommand($"SELECT TOP 0 FROM { quotedTableName };", connection))
{
...
}
}
...
(Or do the dynamic part on SQL Server directly, also using quotename(). But that seems overly and unnecessary tedious, especially if you will do more than one operation on the table in different places.)
Aaron Bertrand's answer solved the problem, but a stored procedure is not useful for a class library that might interact with any database. Here is the way to write RetrieveEmptyDataTable (the method from my question) using his
answer:
private DataTable RetrieveEmptyDataTable(string tableName)
{
const string tableNameParameter = "#TableName";
var query =
" IF EXISTS (SELECT 1 FROM sys.objects\n" +
$" WHERE name = {tableNameParameter})\n" +
" BEGIN\n" +
" DECLARE #sql nvarchar(max) = N'SELECT TOP 0 * \n" +
$" FROM ' + QUOTENAME({tableNameParameter}) + N';';\n" +
" EXEC sys.sp_executesql #sql;\n" +
"END";
using var command = new SqlCommand(query, _connection);
command.Parameters.Add(tableNameParameter, SqlDbType.NVarChar).Value = tableName;
using SqlDataAdapter dataAdapter = new SqlDataAdapter(command);
var table = new DataTable() { TableName = tableName };
Connect();
dataAdapter.Fill(table);
Disconnect();
return table;
}
I need help, I have a problem while inserting a statement in SQL.
I call a SQL statement from my ASP.NET program, some variables contain quotes so when the insert is fired I have an exception like:
Exception Details: System.Data.SqlClient.SqlException: Incorrect syntax near 'xxxxx'.
Unclosed quotation mark after the character string ''.
I don't want the content of my variable to be changed...
Any idea how to handle this?
The C# part :
SqlCommand cmdInsertAssessment = new SqlCommand("xxxxxxx", sqlCnx);
cmdInsertAssessment.CommandType = CommandType.StoredProcedure;
cmdInsertAssessment.Parameters.AddWithValue("#templateID", templateID);
cmdInsertAssessment.Parameters.AddWithValue("#companyID", companyID);
cmdInsertAssessment.Parameters.AddWithValue("#userID",userID);
cmdInsertAssessment.Parameters.AddWithValue("#opn",opn);
cmdInsertAssessment.Parameters.AddWithValue("#mn",Mm);
cmdInsertAssessment.Parameters.AddWithValue("#max",max);
cmdInsertAssessment.Parameters.AddWithValue("#remarque",remarque);
cmdInsertAssessment.Parameters.AddWithValue("#templateTheme",templateTheme);
cmdInsertAssessment.Parameters.AddWithValue("#name", sName);
cmdInsertAssessment.Parameters.AddWithValue("#finished", iFinished);
cmdInsertAssessment.Parameters.AddWithValue("#datenow", dtNow);
try
{
cmdInsertAssessment.ExecuteNonQuery();
}
catch (Exception e)
{
}
SQL part :
CREATE PROCEDURE ["xxxxxxx"] #templateID int,
#companyID int,
#userID int,
#opn nvarchar(255),
#mn nvarchar(255),
#max int,
#remarque nvarchar(255),
#templateTheme nvarchar(255),
#name nvarchar(255),
#finished int,
#datenow datetime
AS
BEGIN
DECLARE
#points AS FLOAT
SET #points=0
IF(#mn='M')
BEGIN
IF(#opn='O')
BEGIN
SET #points=10
END
IF(#opn='P')
BEGIN
SET #points=2
END
END
IF(#mn!='M')
BEGIN
IF(#opn='O')
BEGIN
SET #points=2
END
if(#opn='P')
BEGIN
SET #points=1
END
END
IF(#remarque=NULL)
BEGIN
SET #remarque='nothing'
END
MERGE INTO [dbo].[Assessment] as target
USING (SELECT #templateID,#companyID,#userID,#opn,#points,#max,#remarque,#templateTheme,#datenow,#name,#finished)
As source (_templateID,_companyID,_userID,_opn,_points,_max,_remarque,_templateTheme,_datenow,_name,_finished)
ON target.TemplateID=source._templateID
AND target.TemplateTheme=source._templateTheme
AND target.NameAssessment=source._name
WHEN MATCHED THEN
UPDATE SET Points = source._points, Remarque = source._remarque, FillDate= source._datenow, Finished = source._finished, OPN = source._opn
WHEN NOT MATCHED THEN
INSERT (TemplateID, CompanyID, UserID, OPN, Points, Max, Remarque, TemplateTheme, FillDate, NameAssessment,Finished)
VALUES (source._templateID,source._companyID,source._userID,source._opn,source._points,source._max,source._remarque,source._templateTheme,source._datenow,source._name,source._finished);
END
GO
Thanks :)
Taking things from the begining ! Your procedure calculates a number of points, based on parameters you supply (#mn, #opn), then inserts or updates table Assessment. The first thing to say is that this is not a job for Merge. Merge is intended to operate on two tables, and using it for a row and a table is using a sledgehammer to crack a nut. You should really use
IF EXISTS( SELECT ID FROM ASSESSMENT WHERE... )
then write a classic insert and a classic update. Your procedure will be easier to understand, easier to maintain, and likely run much faster.
If you're still reading, I'll keep going. The calculation of points, business logic that uses nothing from the DB, will be much happier in the C#. Wherever you put it, you can use ternary operators to shorten those either-or choices. The following replaces 20 lines in your procedure.
var points = (mn == 'm')?(opn == 'O'?10:2):(opn == 'O'?2:1);
The assignment starting IF( #remarque = null ) can be done with a null coalescing operator ISNULL() in sql, ?? in C#.
And if you're still still reading, grab QueryFirst. You'll get a v.clean separation between SQL and C# and all your parameter creation will be done for you.
Because you said you wanted to use stored procedures
using (SqlConnection cnn = new SqlConnection(/*Connection String*/))
{
using (SqlCommand cmd = new SqlCommand("MyStoredProcedure", cnn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#param1", "Value 1");
cmd.Parameters.AddWithValue("#param2", "xxxxxx");
cnn.Open();
}
}
Say I have the following SQL statements that I'm executing using ExecuteNonQuery(DbCommand) from C# in a Web Application
DECLARE #InsertedProductID INT -- this is passed as a parameter
DECLARE #GroupID INT -- this is passed as a parameter
DECLARE #total INT
SET #total = (SELECT COUNT (*) FROM Products WHERE GroupID = #GroupID)
UPDATE Products SET ProdName = 'Prod_'+ CAST(#total as varchar(15))
WHERE ProductID = #InsertedProductID
My problem is that I want to ensure that the whole block executes at one. My goal is to always have the ProdName unique per group. If I leave everything the way it is, there is a good chance that I will get duplicate product names if an insert took place in between getting the #total and performing the UPDATE. Is there a way to make sure that the whole SQL block executes at once with no interruption. Will exec or sp_executesql achieve this? My last resort would be to put a lock around the ExecuteNonQuery(DbCommand) But I don't like that since it would create a bottleneck. I don't think that using a sql transaction is helpful here because I'm not worried about the integrity of the commands, I'm rather worried about the parallelism of the commands.
Generally any DML statement (UPDATE/INSERT/DELETE) places a lock (row level / table level) on the particular table but if you want to explicitly guarantee that your operation shouldn't interfere with other executing statement then you should consider placing that entire SQL block inside a transaction block saying
Begin transaction
begin try
DECLARE #InsertedProductID INT -- this is passed as a parameter
DECLARE #GroupID INT -- this is passed as a parameter
DECLARE #total INT
SET #total = (SELECT COUNT (*) FROM Products WHERE GroupID = #GroupID)
UPDATE Products SET ProdName = 'Prod_'+ CAST(#total as varchar(15)) WHERE ProductID = #InsertedProductID
commit; // commits the transaction
end try
begin catch
rollback; //Rolls back the transaction
end catch
end
You should also consider making the Transaction Isolation Level to READ COMMITTED to avoid dirty reads. Also, obviously you should wrap this entire logic in a stored procedure rather executing them as adhoc SQL
If you have control of the creation of your SqlConnection objects, consider relying on database locks using Transactions and an appropriate IsolationLevel. Using Snapshot, for example, will cause the second transaction committed to fail if a separate transaction touched the data before the commit occurred.
Something like:
var c = new SqlConnection(...);
var tran1 = c.BeginTransaction(IsolationLevel.Snapshot);
var tran2 = c.BeginTransaction(IsolationLevel.Snapshot);
DoStuff(c, tran1);//Touch some database data
tran1.Commit();
DoStuff(c, tran2);//Change the same data
tran2.Commit();//Error!
not so sure you could not just do this
UPDATE Products
SET ProdName = 'Prod_'+ CAST((SELECT COUNT (*)
FROM Products
WHERE GroupID = #GroupID) as varchar(15))
WHERE ProductID = #InsertedProductID
But to me that is an odd update
Using a transaction is the right way to go. Along with the other answers, you can also use TransactionScope. The TransactionScope implicitly enrolls the connection and SQL command(s) into a transaction. A rollback will happen automatically if there is an issue since the TransactionScope is in a using block.
Example:
try
{
using (var scope = new TransactionScope())
{
using (var conn = new SqlConnection("your connection string"))
{
conn.Open();
var cmd = new SqlCommand("your SQL here", conn);
cmd.ExecuteNonQuery();
}
scope.Complete();
}
}
catch (TransactionAbortedException ex)
{
}
catch (ApplicationException ex)
{
}
I have an C# method to execute a SQL job. It executes the SQL job successfully.
And the code works perfect.
And I'm using standard SQL stored procedure msdb.dbo.sp_start_job for this.
Here is my code..
public int ExcecuteNonquery()
{
var result = 0;
using (var execJob =new SqlCommand())
{
execJob.CommandType = CommandType.StoredProcedure;
execJob.CommandText = "msdb.dbo.sp_start_job";
execJob.Parameters.AddWithValue("#job_name", "myjobname");
using (_sqlConnection)
{
if (_sqlConnection.State == ConnectionState.Closed)
_sqlConnection.Open();
sqlCommand.Connection = _sqlConnection;
result = sqlCommand.ExecuteNonQuery();
if (_sqlConnection.State == ConnectionState.Open)
_sqlConnection.Close();
}
}
return result;
}
Here is the sp which executing inside the job
ALTER PROCEDURE [Area1].[Transformation]
AS
BEGIN
SET NOCOUNT ON;
SELECT NEXT VALUE FOR SQ_COMMON
-- Transform Master Data
exec [dbo].[sp_Transform_Address];
exec [dbo].[sp_Transform_Location];
exec [dbo].[sp_Transform_Product];
exec [dbo].[sp_Transform_Supplier];
exec [dbo].[sp_Transform_SupplierLocation];
-- Generate Hierarchies and Product References
exec [dbo].[sp_Generate_HierarchyObject] 'Area1',FGDemand,1;
exec [dbo].[sp_Generate_HierarchyObject] 'Area1',RMDemand,2;
exec [dbo].[sp_Generate_Hierarchy] 'Area1',FGDemand,1;
exec [dbo].[sp_Generate_Hierarchy] 'Area1',RMDemand,2;
exec [dbo].[sp_Generate_ProductReference] 'Area1',FGDemand,1;
exec [dbo].[sp_Generate_ProductReference] 'Area1',RMDemand,2;
-- Transform Demand Allocation BOM
exec [Area1].[sp_Transform_FGDemand];
exec [Area1].[sp_Transform_FGAllocation];
exec [Area1].[sp_Transform_RMDemand];
exec [Area1].[sp_Transform_RMAllocation];
exec [Area1].[sp_Transform_BOM];
exec [Area1].[sp_Transform_RMDemand_FK];
-- Transform Purchasing Document Data
exec [dbo].[sp_Transform_PurchasingDoc];
exec [dbo].[sp_Transform_PurchasingItem];
exec [dbo].[sp_Transform_ScheduleLine];
exec [dbo].[sp_CalculateRequirement] 'Area1'
exec [dbo].[sp_Create_TransformationSummary] 'Area1'
-- Trauncate Integration Tables
exec [dbo].[sp_TruncateIntegrationTables] 'Area1'
END
The problem is, even the job is executed successfully or not it always returns -1. How can I identify whether job is successfully executed or not.
After running msdb.dbo.sp_start_job the return code is mapped to an output parameter. You have the opportunity to control the parameter's name prior to execution:
public int StartMyJob( string connectionString )
{
using (var sqlConnection = new SqlConnection( connectionString ) )
{
sqlConnection.Open( );
using (var execJob = sqlConnection.CreateCommand( ) )
{
execJob.CommandType = CommandType.StoredProcedure;
execJob.CommandText = "msdb.dbo.sp_start_job";
execJob.Parameters.AddWithValue("#job_name", "myjobname");
execJob.Parameters.Add( "#results", SqlDbType.Int ).Direction = ParameterDirection.ReturnValue;
execJob.ExecuteNonQuery();
return ( int ) sqlCommand.Parameters["results"].Value;
}
}
}
You need to know the datatype of the return code to do this - and for sp_start_job, it's SqlDbType.Int.
However, this is only the results of starting the job, which is worth knowing, but isn't the results of running your job. To get the results running of your job, you can periodically execute:
msdb.dbo.sp_help_job #jobName
One of the columns returned by the procedure is last_run_outcome and probably contains what you're really interested in. It will be 5 (unknown) while it's still running.
A job is usually the a number of steps - where each step may or may not be executed according to the outcome of previous steps. Another procedure called sp_help_jobhistory supports a lot of filters to specify which specific invocation(s) and/or steps of the job you're interested in.
SQL likes to think about jobs as scheduled work - but there's nothing to keep you from just starting a job ad-hoc - although it doesn't really provide you with much support to correlate your ad-hoc job with an instance is the job history. Dates are about as good as it gets (unless somebody knows a trick I don't know.)
I've seen where the job is created ad-hoc job just prior to running it, so the current ad-hoc execution is the only execution returned. But you end up with a lot of duplicate or near-duplicate jobs laying around that are never going to be executed again. Something you'll have to plan on cleaning up afterwards, if you go that route.
A note on your use of the _sqlConnection variable. You don't want to do that. Your code disposes of it, but it was apparently created elsewhere before this method gets called. That's bad juju. You're better off just creating the connection and disposing of it the same method. Rely on SQL connection pooling to make the connection fast - which is probably already turned on.
Also - in the code you posted - it looks like you started with execJob but switched to sqlCommand - and kinda messed up the edit. I assumed you meant execJob all the way through - and that's reflected in the example.
From MSDN about SqlCommand.ExecuteNonQuery Method:
For UPDATE, INSERT, and DELETE statements, the return value is the number of rows affected by the command. When a trigger exists on a table being inserted or updated, the return value includes the number of rows affected by both the insert or update operation and the number of rows affected by the trigger or triggers. For all other types of statements, the return value is -1. If a rollback occurs, the return value is also -1.
In this line:
result = sqlCommand.ExecuteNonQuery();
You want to return the number of rows affected by the command and save it to an int variable but since the type of statement is select so it returns -1. If you test it with INSERT or DELETE or UPDATE statements you will get the correct result.
By the way if you want to get the number of rows affected by the SELECT command and save it to an int variable you can try something like this:
select count(*) from jobs where myjobname = #myjobname
And then use ExecuteScalar to get the correct result:
result = (int)execJob.ExecuteScalar();
You need to run stored proceedure msdb.dbo.sp_help_job
private int CheckAgentJob(string connectionString, string jobName) {
SqlConnection dbConnection = new SqlConnection(connectionString);
SqlCommand command = new SqlCommand();
command.CommandType = System.Data.CommandType.StoredProcedure;
command.CommandText = "msdb.dbo.sp_help_job";
command.Parameters.AddWithValue("#job_name", jobName);
command.Connection = dbConnection;
using (dbConnection)
{
dbConnection.Open();
using (command){
SqlDataReader reader = command.ExecuteReader();
reader.Read();
int status = reader.GetInt32(21); // Row 19 = Date Row 20 = Time 21 = Last_run_outcome
reader.Close();
return status;
}
}
}
enum JobState { Failed = 0, Succeeded = 1, Retry = 2, Cancelled = 3, Unknown = 5};
Keep polling on Unknown, until you get an answer. Lets hope it is succeeded :-)
Basically I need to check if a table is locked from my code.
I know if I run this directly on the DB;
SELECT * FROM 'tablename' SET LOCK_TIMEOUT 0
I can get an immediate error if a table is locked or none if it isn't. However when I call this statement from my code it still seems to wait for a period of time for the lock to be released before returning. The code I'm using to call this query is;
SqlConnection conn = new SqlConnection(#"ConnectionString");
using(conn)
{
conn.Open();
var query = new SqlCommand("SELECT * FROM tablename SET LOCK_TIMEOUT 0", conn);
try
{
SqlDataReader reader = query.ExecuteReader();
// No lock
}catch (Exception)
{
// Deal with lock exception
}
}
I'd like to know straight away. Any suggestions? Or any better ways of doing this?
It's not clear what goal you are trying to achieve, and for what purpose, but you are setting the lock timeout after the SELECT, whereas you need to set it before.
If you mean transaction lock, check this:
select
object_name(resource_associated_entity_id) as 'TableName' ,*
from
sys.dm_tran_locks
where resource_type = 'OBJECT'
and resource_database_id = DB_ID() and
object_name(resource_associated_entity_id) = N'YourTableName'
This will return no rows if table is not locked by any transaction
Maybe something like this:
select
object_name(resource_associated_entity_id) as 'TableName' ,*
from
sys.dm_tran_locks
where resource_type = 'OBJECT'
and resource_database_id = DB_ID()
This will get you all tables that tables that is looked at the moment