I have an insert statement that was deadlocking using linq. So I placed it in a stored proc incase the surrounding statements were affecting it.
Now the Stored Proc is dead locked. Something about the insert statement is locking itself according to the Server Profiler. It claims that two of those insert statements were waiting for the PK index to be freed:
When I placed the code in the stored procedure it is now stating that this stored proc has deadlocked with another instance of this stored proc.
Here is the code. The select statement is similar to that used by linq when it did its own query. I simply want to see if the item exists and if not then insert it. I can find the system by either the PK or by some lookup values.
SET NOCOUNT ON;
BEGIN TRY
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRANSACTION SPFindContractMachine
DECLARE #id int;
set #id = (select [m].pkID from Machines as [m]
WHERE ([m].[fkContract] = #fkContract) AND ((
(CASE
WHEN #bByID = 1 THEN
(CASE
WHEN [m].[pkID] = #nMachineID THEN 1
WHEN NOT ([m].[pkID] = #nMachineID) THEN 0
ELSE NULL
END)
ELSE
(CASE
WHEN ([m].[iA_Metric] = #lA) AND ([m].[iB_Metric] = #lB) AND ([m].[iC_Metric] = #lC) THEN 1
WHEN NOT (([m].[iA_Metric] = #lA) AND ([m].[iB_Metric] = #lB) AND ([m].[iC_Metric] = #lC)) THEN 0
ELSE NULL
END)
END)) = 1));
if (#id IS NULL)
begin
Insert into Machines(fkContract, iA_Metric, iB_Metric, iC_Metric, dteFirstAdded)
values (#fkContract, #lA, #lB, #lC, GETDATE());
set #id = SCOPE_IDENTITY();
end
COMMIT TRANSACTION SPFindContractMachine
return #id;
END TRY
BEGIN CATCH
if ##TRANCOUNT > 0
ROLLBACK TRANSACTION SPFindContractMachine
END CATCH
Any procedure that follows the pattern:
BEGIN TRAN
check if row exists with SELECT
if row doesn't exist INSERT
COMMIT
is going to run into trouble in production because there is nothing to prevent two treads doing the check simultaneously and both reach the conclusion that they should insert. In particular, under serialization isolation level (as in your case), this pattern is guaranteed to deadlock.
A much better pattern is to use database unique constraints and always INSERT, capture duplicate key violation errors. This is also significantly more performant.
Another alternative is to use the MERGE statement:
create procedure usp_getOrCreateByMachineID
#nMachineId int output,
#fkContract int,
#lA int,
#lB int,
#lC int,
#id int output
as
begin
declare #idTable table (id int not null);
merge Machines as target
using (values (#nMachineID, #fkContract, #lA, #lB, #lC, GETDATE()))
as source (MachineID, ContractID, lA, lB, lC, dteFirstAdded)
on (source.MachineID = target.MachineID)
when matched then
update set #id = target.MachineID
when not matched then
insert (ContractID, iA_Metric, iB_Metric, iC_Metric, dteFirstAdded)
values (source.contractID, source.lA, source.lB, source.lC, source.dteFirstAdded)
output inserted.MachineID into #idTable;
select #id = id from #idTable;
end
go
create procedure usp_getOrCreateByMetrics
#nMachineId int output,
#fkContract int,
#lA int,
#lB int,
#lC int,
#id int output
as
begin
declare #idTable table (id int not null);
merge Machines as target
using (values (#nMachineID, #fkContract, #lA, #lB, #lC, GETDATE()))
as source (MachineID, ContractID, lA, lB, lC, dteFirstAdded)
on (target.iA_Metric = source.lA
and target.iB_Metric = source.lB
and target.iC_Metric = source.lC)
when matched then
update set #id = target.MachineID
when not matched then
insert (ContractID, iA_Metric, iB_Metric, iC_Metric, dteFirstAdded)
values (source.contractID, source.lA, source.lB, source.lC, source.dteFirstAdded)
output inserted.MachineID into #idTable;
select #id = id from #idTable;
end
go
This example separates the two cases, since T-SQL queries should never attempt to resolve two different solutions in one single query (the result is never optimizable). Since the two tasks at hand (get by mahcine id and get by metrics) are completely separate, the should be separate procedures and the caller should call the apropiate one, rather than passing a flag. This example shouws how to achieve the (probably) desired result using MERGE, but of course, a correct and optimal solution depends on the actual schema (table definition, indexes and cosntraints in place) and on the actual requirements (is not clear what the procedure is expected to do if the criteria is already matched, not output and #id?).
By eliminating the SERIALIZABLE isolation, this is no longer guaranteed to deadlock, but it may still deadlock. Solving the deadlock is, of course, completely dependent on the schema which was not specified, so a solution to the deadlock cannotactually be provided in this context. There is a sledge hammer of locking all candidate row (force UPDLOCK or even TABLOCX) but such a solution would kill throughput on heavy use, so I cannot recommend it w/o knowing the use case.
Get rid of the transaction. It's not really helping you, instead it is hurting you. That should clear up your problem.
How about this SQL? It moves the check for existing data and the insert into a single statement. This way, when two threads are running they're not deadlocked waiting for each other. At best, thread two is deadlocked waiting for thread one, but as soon as thread one finishes, thread two can run.
BEGIN TRY
BEGIN TRAN SPFindContractMachine
INSERT INTO Machines (fkContract, iA_Metric, iB_Metric, iC_Metric, dteFirstAdded)
SELECT #fkContract, #lA, #lB, #lC, GETDATE()
WHERE NOT EXISTS (
SELECT * FROM Machines
WHERE fkContract = #fkContract
AND ((#bByID = 1 AND pkID = #nMachineID)
OR
(#bByID <> 1 AND iA_Metric = #lA AND iB_Metric = #lB AND iC_Metric = #lC))
DECLARE #id INT
SET #id = (
SELECT pkID FROM Machines
WHERE fkContract = #fkContract
AND ((#bByID = 1 AND pkID = #nMachineID)
OR
(#bByID <> 1 AND iA_Metric = #lA AND iB_Metric = #lB AND iC_Metric = #lC)))
COMMIT TRAN SPFindContractMachine
RETURN #id
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
ROLLBACK TRAN SPFindContractMachine
END CATCH
I also changed those CASE statements to ORed clauses just because they were easier to read to me. If I recall my SQL theory, ORing might make this query a little slower.
I wonder if adding an UPDLOCK hint to the earlier SELECT(s) would fix this; it should avoid sone deadlock scenarios be preventing another spud getting a read-lock on the data you are about to mutate.
Related
We have a backend job written in C# with threading and generate a new thread in every second (Somehow, we can't increase this time. ).
It's read data from DB for processing help of a stored procedure and send request to interfacing system.
Currently we are facing issue where same data is pulled in multiple process and found deadlock in our log table. Please suggest how can we impalement locking so the same type of data can be processed by only a single process and other process will have the different data.
DB: SQL Server
SP Code: Given below
ALTER PROCEDURE [Migration]
AS
BEGIN
declare #ConversationID varchar(200)='',Group varchar(100) =''
-- Select records with Flag 0
select
top 1 #ConversationID = ConversationID,
#Group = Group
from [Migration]
where NewCode = (select top 1 NewCode
from [Migration]
where Flag = 0 group by NewCode, Group, InsertDate
)
and Flag = 0;
select * from [Migration] where ConversationID = #ConversationID and Group = #Group;
BEGIN TRANSACTION
BEGIN TRY
update [Migration] set Flag = 1 where ConversationID = #ConversationID and Group = #Group and
Flag = 0;
COMMIT TRANSACTION
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION
insert into Logs(ErrorType,Description) values('MigrationError',ERROR_MESSAGE());
END CATCH
END
The typical deadlock message is something like
Transaction (Process ID 100) was deadlocked on lock resources with
another process and has been chosen as the deadlock victim. Rerun the
transaction.
However, it's hard to see where your query would get deadlocked the traditional way - I can see it being blocked, but not deadlocked, as you typically need more than one update to be occurring within the same transaction. Indeed, your transaction above is only a single command - the transaction around it is fairly meaningless as each command is run in its own implicit transaction anyway.
Instead, I'm guessing that your deadlock message is, instead, something like
Transaction (Process ID 100) was deadlocked on lock | communication
buffer resources with another process and has been chosen as the
deadlock victim. Rerun the transaction.
Note the difference in what the deadlock was on - in this case, the lock/communication buffer resources.
These deadlocks are related to issues with parallelism; see (for example) What does "lock | communication buffer resources" mean? for more information.
You will tend to get this if the SP does parallel processing and you're running this SP many times in a row. Setting MAXDOP 1 and/or improving your indexing often fixes such issues - but obviously you will need to do your research on the best approach in your own specific circumstances.
Regarding the question itself - how to make it so that only one thing can deal with a given row at a time?
There are numerous methods. My usual methods involve statements with OUTPUT clauses. The advantage of those is that you can do a data insert/update as well as record what you have inserted or updated within the same command.
Here is an example from your current thing - it sets the flag to be -1 for the rows it's dealing with.
CREATE TABLE #ActiveMigrations (ConversationID varchar(200), CGroup varchar(100));
-- Select records with Flag 0
WITH TopGroup AS (
select top 1 ConversationID, CGroup, Flag
from [Migration]
where NewCode = (select top 1 NewCode
from [Migration]
where Flag = 0 group by NewCode, CGroup, InsertDate
)
and Flag = 0
)
UPDATE TopGroup
SET Flag = -1
OUTPUT ConversationID, CGroup
INTO #ActiveMigrations (ConversationID, CGroup)
FROM TopGroup;
In the above, you have the ConversaionID and Group in the temporary table (rather than variables) for further processing, and the flags set to -1 so they're not picked up by further processing.
A similar version can be used to track (in a separate table) which are being operated on. In this case, you create a scratch table that includes active conversations/groups e.g.,
-- Before you switch to the new process, create the table as below
CREATE TABLE ActiveMigrations (ConversationID varchar(200), CGroup varchar(100));
You can code this with OUTPUT clauses as above, and (say) include this table in your initial SELECT as a LEFT OUTER JOIN to exclude any active rows.
A simpler version is to use your SELECT as above to get a #ConversationID and #CGroup, then try to insert it the table
INSERT INTO ActiveMigrations (ConversationID, CGroup)
SELECT #ConversationID, #Group
WHERE NOT EXISTS
(SELECT 1 FROM ActiveMigrations WHERE ConversationID = #ConversationID AND CGroup = #Group);
IF ##ROWCOUNT = 1
BEGIN
...
The key thing to think about with these, is if you have the command running twice simultaneously, how could they interact badly?
To be honest, your code itself
update [Migration]
set Flag = 1
where ConversationID = #ConversationID
and Group = #Group
and Flag = 0;
protects itself because a) it's one command only, and b) it has the Flag=0 protection on the end.
It does waste resources as the second concurrent process does all the work, then gets to the end and has nothing to do as the first process already updated the row. For something running regularly and likely to have concurrency issues, then you probably should code it better.
I have a stored procedure which takes #id as input parameter. Table Student has primary key on Id_Column and Count_Column. If any record is present in table for the given id, then I select the max Count_Column from table and insert new row by incrementing max Count_Column by 1 else with zero value.
I'm calling this stored procedure from ado.net code in a WCF service and this service is called from an asp.net web application.
This stored procedure works fine in normal cases, but when multiple users calls it at same time, primary key violation issue happen, same case I have reproduced by making application multithreaded too. I know that this issue is related to concurrency, initially I was using with(nolock) in select query, but I have removed this now.
Somewhere I have read that by setting transaction isolation level it can be solved but when I tried I am some getting rollback transaction exception.
Please let me know any efficient solution for this problem.
declare #iCount = 0;
if exists(select 'x' from Student with(nolock) where Id_Column = #iId)
begin
set #iCount = (select max(Count_Column)
from Student
where Id_Column = #iId)
end
insert into Student
values(#id, #iCount + 1);
2nd solution:
begin try
set transaction isolation level serializable
begin transaction
declare #iCount = 0;
if exists(select 'x' from from Student with(nolock) where Id_Column = #iId)
begin
set #iCount = (select max(Count_Column)
from Student
where Id_Column = #iId)
end
insert into Student
values(#id, #iCount + 1);
commit transaction
end try
begin catch
rollback transaction
end catch
Try something like......
BEGIN TRY
BEGIN TRANSACTION;
DECLARE #iCount INT;
IF EXISTS(SELECT 1 FROM Student WITH(UPDLOCK,HOLDLOCK) WHERE Id_Column = #iId)
BEGIN
select #iCount = ISNULL(max(Count_Column), 0) + 1
from Student WITH(UPDLOCK,HOLDLOCK) where Id_Column = #iId
insert into Student
values(#id, #iCount);
END
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
IF (##TRANCOUNT <> 0)
ROLLBACK TRANSACTION;
END CATCH
Important Note
You should really be using Identity column here to handle the auto-increment value. If you are using sql server 2012 or later you also have another option of using Sequence Object also an auto-increment .
There is a failsafe strategy: SERIALIZABLE isolation and retry in case of deadlock. The definition of SERIALIZABLE is as-if-serial execution which excludes all race conditions.
I don't know why you are catching exceptions there. All that accomplishes is suppress errors.
Oracle has sequences and MS followed with Sql Server 2012 with their implementation calling it sequence objects which is not table specific.
CREATE SEQUENCE
Schema.MySequence
AS int
INCREMENT BY 1 ;
You can then use it as follow:
DECLARE #iCount int ;
SET #iCount = NEXT VALUE
FOR Schema.MySequence;
This is similar as the nextval in Oracle and will ensure uniqueness. The only question is if you will accept gaps in your count sequence in the case of rolledback/failed transactions or not...
I am using INSERT Trigger on that table. Once trigger is executed (it update the table if a condition is meet), that is where the problem is.
int records = sc.ExecuteNonQuery(); // works ok if trigger does not update the record
The above code always ruturns -1 if I leave SET NOCOUNT ON; in the stored procedure itself. If I remove it, I get correct result but if trigger does update the record, then wrong result. I sometime get 10 or a different number. My Trigger looks like this
UPDATE students
SET status = 'Inactive'
FROM Inserted i
INNER JOIN students T2
ON i.sname = T2.sname
AND i.id <> t2.id
That means it can return more than one record (esp in test cases). Can someone tell me what is the cure? I am open to use Functions if that solves the problem or any better approach.
Thanks
Adding Insert SP
CREATE PROCEDURE sp_InsertSudent
-- Add the parameters for the stored procedure here
#student_name varchar(25) = null,
#status varchar(20) = null,
#renew varchar(15) = null,
#edate datetime = null
AS
BEGIN
--SET NOCOUNT ON;
insert into students VALUES(#student_name,#status,#renew,#edate)
END
GO
Note: I am looking for an error because the fields are picked from Excel. if any field is in wrong format or empty, the Insert SP will produce error. I must convey that error to the user.
Adding Actual SP
So the whole problem is in the SP. If I remove it, everything works fine. Here is my actual SP
UPDATE CustomerTbl
SET [Account Status] = 'Inactive',
[End Date] = DateAdd(day,-1,[Supplier End Date]),
[EnrollmentStatus] = 'Waiting'
WHERE OET.[Account No] = (SELECT [Account No] FROM CustomerTbl WHERE id = ##identity)
AND id <> ##identity
The logic is the same as above but stated differently. The ExecuteNonQuery oupts the result of this trigger than than the actual storedprocedure, so what is he cure? Can suppress its output somehow.
I would add Try Catch blocks to the proc and have it return 1 if successful and 0 if not successful.
I also would adjust the trigger to be more efficient by only chaning the status of those where are active and meet the other criteria.
UPDATE students
SET status = 'Inactive'
FROM Inserted i
INNER JOIN students T2
ON i.sname = T2.sname
AND i.id <> t2.id
AND status <> 'inactive'
This could save you from updating 1000 rows when you only really need to update one active row.
My own answer (not complete yet). According to MSDN documentation for ExecuteNonQuery
When a trigger exists on a table being inserted or updated, the return
value includes the number of rows affected by both the insert or
update operation and the number of rows affected by the trigger or
triggers.
This means I need to modify the trigger itself to accommodate the logic, or even by the fact that when trigger is called, that proves that a record was successful. That means if I get anything greater than 0, that should be assumed as success. Although not solid logic but it will work. SET COUNT ON must be commented for this.
maybe I'm not understanding the question, but if your goal is to track the number of rows inserted, then you would maintain a count of how many times you're calling your stored procedure (assuming your sproc inserts one row at a time, of course).
alternately, if you are trying to maintain a count of the total # of records affected (by both inserts and updates), then you could incorporate the logic in the trigger into your sproc. the rows affected by both the insert statement (always 1) and by the update statement (##rowcount after update completes) ... you could then return that value to the caller.
UPDATE:
Sure, return 0 if there is an error. If using SQL2005 (or above), use TRY/CATCH mechanism.
Add an extra column that the stored proc doesnt populate (leaves null)
then when the trigger runs, simply update the Null values to a non-null value and use the RowCount from the Update to determine how many rows were updated.
We have the ability to execute stored procs from the middle-tier. Basically, in a database table called "SQLJobStep" we have -- among other things -- a varchar(300) column called "StoredProcedure" which holds something like this:
usp_SendReminderEmails #Debug=0, #LoginID
Via the middle-tier, a user clicks on a link to run the proc of their choice. What happens in the backend is a proc ("usp_SQLJobsActionGet") looks up the correct value from the "SQLJobStep.StoredProcedure" column and executes the value above.
This is the part of the code from "usp_SQLJobsActionGet" that executes the above value:
DECLARE #StepId int
DECLARE #LoginID varchar(12)
DECLARE #StoredProcedure varchar(300)
SET #StepId = 74
SET #LoginID = 'Yoav'
SELECT #StoredProcedure = SJS.StoredProcedure
FROM SQLJobStep AS SJS
WHERE SJS.StepId = #StepId
SET #StoredProcedure = ISNULL(#StoredProcedure, '')
IF CHARINDEX('#LoginID', #StoredProcedure) > 0
BEGIN
SET #LoginID = ISNULL(#LoginID, 'UNKNOWN')
SET #StoredProcedure = REPLACE(#StoredProcedure, '#LoginID', '#LoginID = ''' + #LoginID + '''')
END
IF #StoredProcedure != ''
BEGIN
EXEC(#StoredProcedure)
END
Fairly simple stuff....
The above code converts the original value to (and then executes):
usp_SendReminderEmails #Debug=0, #LoginID = 'Yoav'
Here is the issue:
When executing the "usp_SendReminderEmails #Debug=0, #LoginID = 'Yoav'" value nothing happens. No error is returned to the middle-tier. But I know that a value is pulled from the SQLJobStep table because we have other stored procedure values that get pulled and they run fine. (Note that the other values only have the #LoginID parameter, while this has #Debug=0 as well.)
At the same time, if I run the code above (both the gutted code and calling "usp_SQLJobsActionGet" directly) in SQL Management Studio, it works perfectly.
Do you have any advice? I am sure I am missing something very basic.
Thanks.
My advice? Use sp_ExecuteSQL instead of concatenation / replacement:
IF #StoredProcedure != ''
BEGIN
EXEC sp_ExecuteSQL #StoredProcedure, N'#LoginID varchar(12)', #LoginID
END
Overall, though - the EXEC should work; are you sure that #StoredProcedure is not empty?
Thanks for helping. I found the answer to my issue, and as you can guess it had to do with issues beyond what I described originally:
In the usp_SendReminderEmails proc, we call another proc in order to audit each e-mail record that is sent. This auditing proc inserts a record into an audit table and then returns the identity (SELECT TOP 1 SCOPE_IDENTITY()). While it only returns 1 record at a time, it happens to be called in a cursor (in usp_SendReminderEmails) to send out each email at a time (note: this is a SQL Job proc).
What I noticed is that upon executing usp_SendReminderEmails #Debug=0, #LoginID = 'Yoav' in Management Studio, it works fine but there is a warning returned(!):
The query has exceeded the maximum number of result sets that can be displayed in the results grid. Only the first 100 result sets are displayed in the grid.
When calling the proc from the middle-tier, therefore, nothing happened - no error returned, but no processing of usp_SendReminderEmails either. I fixed it by calling the audit proc in an insert into temp table in usp_SendReminderEmails, thereby ensuring it doesn't get returned (since it's only an identity value):
INSERT INTO #AuditData (AuditDataId)
EXEC usp_AuditSave
Here is my current implementation of a stored procedure which returns Order status for a given Order ID. There are two situations,
there is matched Order ID and I will retrieve the related status,
there is no matched Order ID (i.e. non-existing Order ID).
My confusion is, how to implement the two functions elegantly/efficiently in one stored procedure, so that I return matched Order ID for situation 1 and also indicate client no matched Order ID in situation 2?
I am using VSTS 2008 + C# + ADO.Net + .Net 3.5 as client, and using SQL Server 2008 as server.
CREATE PROCEDURE [dbo].[GetStatus]
#ID [nvarchar](256),
#Status [int] output
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
SELECT #Status = [Status]
FROM [dbo].[OrderStatus]
WHERE (#ID = [ID]);
END
thanks in advance,
George
why are you using output parameter.
you just need to take your stored procedure result in dataset of the data access layer.
just check that if (dataset != null) then take value else return appropriate message to your business layer.
There are multiple approaches you can take:
Keep everything as is and in your .NET code, if the #status value returned is DBNull, then it will indicate situation 2, otherwise situation 1.
Add a RETURN statement to the SPROC and use
Dim returnValue As New SqlParameter("#RETURN_VALUE", SqlDbType.Int)
returnValue.Direction = ParameterDirection.ReturnValue
Cmd.Parameters.Add(returnValue)
in your .NET code to explicitly identify what the SPROC returned and take action accordingly.
As an additional tip, use a SET instead of SELECT when assigning the value to #Status variable in the SPROC. This will guarantee that you get a NULL back if there is no match found. So,
`
-- Insert statements for procedure here
SET #Status = SELECT [Status]
FROM [dbo].[OrderStatus]
WHERE (#ID = [ID]);`
You can use the "if statements" inside the stored procedures. the web site at bottom gives you some tips.
http://translate.google.com.br/translate?u=http%3A%2F%2Fmail.firebase.com.br%2Fpipermail%2Flista_firebase.com.br%2F2005-November%2F021883.html&sl=pt&tl=en&hl=pt-BR&ie=UTF-8