I have over a million records in the list. I pass all records at once from table to stored procedure .In stored procedure i have to have iteration to go thorugh all the rows in the table and for each row it takes table row modified date based on jobid and checks if it exist in database and based on it either it updates or insert the record. I feel that my procedure is not correct, would be glad if someone help on this.
foreach (No_kemi no_list in newforSQL)
{
DataTable _dt = new DataTable("table");
_dt.Columns.Add("JobID", typeof(string));
_dt.Columns.Add("CreatedDate", typeof(datetime));
_dt.Columns.Add("ModifiedDate", typeof(datetime));
_dt.Columns.Add("DbDate", typeof(datetime));
_dt.Columns.Add("SubGUID", typeof(string));
_dt.Columns.Add("eType", typeof(string));
// adding over a million records in the table
_dt.Rows.Add(no_list.ID,no_list.CreatedDate,no_list.ModifiedDate,no_list.DbDate,no_list.SubGUID,no_list.eType);
}
using (SqlCommand sqlCommand = new SqlCommand())
{
sqlCommand.CommandType = CommandType.StoredProcedure;
sqlCommand.CommandText = "Process_NO_table";
sqlCommand.Connection = connection;
SqlParameter typeParam = sqlCmd.Parameters.AddWithValue("#track", _dt);
typeParam .SqlDbType = SqlDbType.Structured;
sqlCmd.ExecuteNonQuery();
}
my tabletype and procedure:
CREATE TYPE TrackType AS TABLE
(
t_Id uniqueidentifier, t_JobID nvarchar(50), t_CreatedDate datetime2(7), t_ModifiedDate datetime2(7), t_DbDate datetime2(7)
t_SubGUID nvarchar(MAX), t_eType nvarchar(MAX)
);
GO
ALTER/CREATE PROCEDURE [dbo].[Process_NO_table] // i will change to alter after i create it
#track TrackType READONLY
AS
// i need to iterate all the rows of the table(over a million)
Declare #rows INT
Declare #i int = 0
Declare #count int = (SELECT COUNT(*) FROM #track)
DECLARE #is INT
WHILE (#i < #count)
BEGIN
-- first i check modified date from the database table
SELECT #is = COUNT(*) FROM NO_table WHERE [JobID] IN (SELECT [t_JobID] FROM #track)
MERGE [dbo].[NO_table] AS [Target]
USING #track AS [Source]
-- if the database modifed date is less than the modifeid date from the proceduretable(#track) then it updates the records
ON [Target].[ModifiedDate] < [Source].[t_ModifiedDate] AND JobID = t_JobID
WHEN MATCHED THEN
UPDATE SET [JobID] = [Source].[t_JobID],
[CreatedDate] = [Source].[t_CreatedDate]
[DbDate]= [Source].[t_DbDate]
[ModifiedDate] = [Source].[t_ModifiedDate]
[SubGUID] = [Source].[t_SubGUID]
[eType] = [Source].[t_eType]
-- if the database modifed dateis not existing then it insert the record
MERGE [dbo].[NO_table] AS [Target]
USING #track AS [Source]
ON (#is != 0)
WHEN NOT MATCHED THEN
INSERT INTO [NO_table] ( [JobID], [CreatedDate], [ModifiedDate], [DbDate], [SubGUID], [eType] )
VALUES ( [Source].[t_JobID], [Source].[t_CreatedDate], [Source].[t_ModifiedDate], [Source].[t_DbDate], [Source].[t_SubGUID], [Source].[t_eType] );
SET #i = #i + 1
END
GO
I think you have a large number of syntax errors in your SQL (assuming MS SQL), but your merge condition is probably giving you the invalid syntax near WHERE, because you need to use AND, not WHERE.
ON [Target].[ModifiedDate] < [Source].[t_ModifiedDate] WHERE JobID = t_JobID
should be
ON [Target].[ModifiedDate] < [Source].[t_ModifiedDate] AND JobID = t_JobID
The Select Top 1 and the WHEN MATCHED THEN after the null check for #dbmoddate need to go away as well, as those are also causing syntax issues.
The insert after the null check for #dbmoddate needs a table specified so it actually knows what to insert into.
You also need to end your merge statement with a semicolon.
UPDATED ANSWER:
Now that you have this more cleaned up, I can better see what you're trying to do. At a high level, you want to simply update existing records where the modified date is less than the modified date of on your custom type. If there does not exist a record in your table that does exist in your custom type, then insert it.
With that said, you don't actually need to loop because you aren't doing anything with your loop. What you currently have and what I'm posting below this is all set-based results, not iterative.
You can make this much simpler by getting rid of the merge statements and doing a simple Update and Insert like I have below. The merge would make more sense if your condition between the two statements was the same (i.e. if you didn't have the check for modified date, then merge would be OK) because then you can use the keywords WHEN MATCHED and WHEN NOT MATCHED and have it in one single merge statement. I personally stay away from MERGE statements because they tend to be a little buggy and there are a number of things you have to watch out for.
I think this solution will be better in the long run as it is easier to read and more maintainable...
CREATE TYPE TrackType AS TABLE
(
t_Id uniqueidentifier, t_JobID nvarchar(50), t_CreatedDate datetime2(7), t_ModifiedDate datetime2(7), t_DbDate datetime2(7)
,t_SubGUID nvarchar(MAX), t_eType nvarchar(MAX)
);
GO
CREATE PROCEDURE [dbo].[Process_NO_table] -- i will change to alter after i create it
#track TrackType READONLY
AS
-- i need to iterate all the rows of the table(over a million)
Update [NO_table]
SET [JobID] = T.[t_JobID],
[CreatedDate] = T.[t_CreatedDate],
[DbDate]= T.[t_DbDate],
[ModifiedDate] = T.[t_ModifiedDate],
[SubGUID] = T.[t_SubGUID] ,
[eType] = T.[t_eType]
From #track T
Where [NO_table].[JobID] = T.[t_JobID]
And [NO_table].[ModifiedDate] < T.[t_ModifiedDate]
Insert [NO_Table]
(
[JobID],
[CreatedDate],
[ModifiedDate],
[DbDate],
[SubGUID],
[eType]
)
Select T.[t_JobID],
T.[t_CreatedDate],
T.[t_ModifiedDate],
T.[t_DbDate],
T.[t_SubGUID],
T.[t_eType]
From #track T
Where Not Exists (Select 1 From [NO_table] where T.[t_JobID] = [NO_table].[JobID])
GO
Related
I want to run an UPDATE SQL query within a C# foreach loop, such as:
var alterQuery = $#"
IF NOT EXISTS(SELECT 1 FROM sys.columns WHERE Name = 'MyColumn' AND Object_ID = Object_ID('MyTable'))
ALTER TABLE MyTable
ADD [MyColumn] nvarchar(255)
";
using (var connection = myConnection)
{
connection.Execute(alterQuery);
foreach (var obj in myObjects)
{
var query = $#"UPDATE [MyTable]
SET [MyColumn] = '{obj.Val}'
WHERE [ID] = '{obj.ID}'
";
// note: my Execute method uses ExecuteNonQuery() behind the scenes
connection.Execute(query);
}
}
But, I receive the following SQL Exception. I receive it when there are two or more values in myObjects but not when there is only one:
Additional information: Column names in each table must be unique. Column name MyColumn in table MyTable is specified more than once.
I believe it may be due to the queries running simultaneously and trying to access the same column (MyColumn). Should I be running my queries in such a way that each must wait until the previous completes?
How can I successfully run these queries?
I solved this issue by adding BEGIN and END around my ALTER TABLE query.
Does not work:
IF NOT EXISTS(SELECT 1 FROM sys.columns WHERE Name = 'MyColumn' AND
Object_ID = Object_ID('MyTable'))
ALTER TABLE MyTable
ADD [MyColumn] nvarchar(255)
Works:
IF NOT EXISTS(SELECT 1 FROM sys.columns WHERE Name = 'MyColumn' AND
Object_ID = Object_ID('MyTable'))
BEGIN
ALTER TABLE MyTable
ADD [MyColumn] nvarchar(255)
END
I find it weird that in the non-working case, the only line that the IF NOT EXISTScheck controls is the ALTER TABLE MyTable line, rather than both the ALTER TABLE MyTable and the ADD [MyColumn] nvarchar(255) lines, which should be interpreted as one statement.
I have a Gridview in front end where Grid have two columns : ID and Order like this:
ID Order
1 1
2 2
3 3
4 4
Now user can update the order like in front end Gridview:
ID Order
1 2
2 4
3 1
4 3
Now if the user click the save button the ID and order data is being sent to Stored Procedure as #sID = (1,2,3,4) and #sOrder = (2,4,1,3)
Now if I want to update the order and make save I want to store it into database. Through Stored procedure how can update into the table so that the table is updated and while select it gives me the results like:
ID Order
1 2
2 4
3 1
4 3
There is no built in function to parse these comma separated string. However, yo can use the XML function in SQL Server to do this. Something like:
DECLARE #sID VARCHAR(100) = '1,2,3,4';
DECLARE #sOrder VARCHAR(10) = '2,4,1,3';
DECLARE #sIDASXml xml = CONVERT(xml,
'<root><s>' +
REPLACE(#sID, ',', '</s><s>') +
'</s></root>');
DECLARE #sOrderASXml xml = CONVERT(xml,
'<root><s>' +
REPLACE(#sOrder, ',', '</s><s>') +
'</s></root>');
;WITH ParsedIDs
AS
(
SELECT ID = T.c.value('.','varchar(20)'),
ROW_NUMBER() OVER(ORDER BY (SELECT 1)) AS RowNumber
FROM #sIDASXml.nodes('/root/s') T(c)
), ParsedOrders
AS
(
SELECT "Order" = T.c.value('.','varchar(20)'),
ROW_NUMBER() OVER(ORDER BY (SELECT 1)) AS RowNumber
FROM #sOrderASXml.nodes('/root/s') T(c)
)
UPDATE t
SET t."Order" = p."Order"
FROM #tableName AS t
INNER JOIN
(
SELECT i.ID, p."Order"
FROM ParsedOrders p
INNER JOIN ParsedIDs i ON p.RowNumber = i.RowNumber
) AS p ON t.ID = p.ID;
Live Demo
Then you can put this inside a stored procedure or whatever.
Note that: You didn't need to do all of this manually, it should be some way to make this gridview update the underlying data table automatically through data binding. You should search for something like this instead of all this pain.
You could use a table valued parameter to avoid sending delimiter-separated values or even XML to the database. To do this you need to:
Declare a parameter type in the database, like this:
CREATE TYPE UpdateOrderType TABLE (ID int, Order int)
After that you can define the procedure to use the parameter as
CREATE PROCEDURE UpdateOrder (#UpdateOrderValues UpdateOrderType readonly)
AS
BEGIN
UPDATE t
SET OrderID = tvp.Order
FROM <YourTable> t
INNER JOIN #UpdateOrderValues tvp ON t.ID=tvp.ID
END
As you can see, the SQL is trivial compared to parsing XML or delimited strings.
Use the parameter from C#:
using (SqlCommand command = connection.CreateCommand()) {
command.CommandText = "dbo.UpdateOrder";
command.CommandType = CommandType.StoredProcedure;
//create a table from your gridview data
DataTable paramValue = CreateDataTable(orderedData)
SqlParameter parameter = command.Parameters
.AddWithValue("#UpdateOrderValues", paramValue );
parameter.SqlDbType = SqlDbType.Structured;
parameter.TypeName = "dbo.UpdateOrderType";
command.ExecuteNonQuery();
}
where CreateDataTable is something like:
//assuming the source data has ID and Order properties
private static DataTable CreateDataTable(IEnumerable<OrderData> source) {
DataTable table = new DataTable();
table.Columns.Add("ID", typeof(int));
table.Columns.Add("Order", typeof(int));
foreach (OrderData data in source) {
table.Rows.Add(data.ID, data.Order);
}
return table;
}
(code lifted from this question)
As you can see this approach (specific to SQL-Server 2008 and up) makes it easier and more formal to pass in structured data as a parameter to a procedure. What's more, you're working with type safety all the way, so much of the parsing errors that tend to crop up in string/xml manipulation are not an issue.
You can use charindex like
DECLARE #id VARCHAR(MAX)
DECLARE #order VARCHAR(MAX)
SET #id='1,2,3,4,'
SET #order='2,4,1,3,'
WHILE CHARINDEX(',',#id) > 0
BEGIN
DECLARE #tmpid VARCHAR(50)
SET #tmpid=SUBSTRING(#id,1,(charindex(',',#id)-1))
DECLARE #tmporder VARCHAR(50)
SET #tmporder=SUBSTRING(#order,1,(charindex(',',#order)-1))
UPDATE dbo.Test SET
[Order]=#tmporder
WHERE ID=convert(int,#tmpid)
SET #id = SUBSTRING(#id,charindex(',',#id)+1,len(#id))
SET #order=SUBSTRING(#order,charindex(',',#order)+1,len(#order))
END
I have three tables tb1,tb2 and tbTotal. They have the same schemas. The tables have three columns, MetricID, Descr and EntryDE.
What I want is to merge tb1 with tbTotal. I have done this and it works fines.
My stored procedure is:
CREATE PROCEDURE [dbo].[Admin_Fill]
-- Add the parameters for the stored procedure here
#MetricId INT,
#Descr VARCHAR(100),
#EntryDE VARCHAR(20)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
--SET IDENTITY_INSERT dbo.tbTotal ON
-- Insert statements for procedure here
;WITH cte AS (SELECT MetricId=#MetricId,Descr=#Descr,EntryDE=#EntryDE)
MERGE tbTotal d
USING cte s
ON s.EntryDE = d.EntryDE
AND s.MetricId=d.MetricId
WHEN matched THEN UPDATE
set MetricId=s.MetricId,
Descr=s.Descr,
EntryDE=s.EntryDE
WHEN not matched BY TARGET THEN
INSERT(MetricId,Descr,EntryDE)
VALUES (s.MetricId,s.Descr,s.EntryDE);
END
My C# code:
foreach (DataRow row in dt.Rows) // pass datatable dt1
{
MetricId = Convert.ToInt32(row["MetricId"]);
Descr = row["Descr"].ToString();
EntryDE = row["EntryDE"].ToString();
parameters.Add("#MetricId", MetricId);
parameters.Add("#Descr", Descr);
parameters.Add("#EntryDE", EntryDE);
dbaccess.ExecuteNonQuery(strStoredProcedure, parameters); //cmd.ExecuteNonQuery();
parameters.Clear();
}
Also I want to remove all records in dt2 from dtTotal. I am not sure how to modify the stored procedure.
Thanks for help.
If i have understood what you are trying to do correctly, then this is potentially how I would prefer to implement the solution.
I would pass the 2 datatables as TABLE variables to an SP - similar to below and then use JOINs to both UPDATE and DELETE as required using SET operations - thus affecting multiple rows in one query and avoid looping through each row separately.
As mentioned by AdaTheDev in the related answer, you will end up creating a "TABLE" type but there is no drawback to having one extra type and this solution will scale a lot better than a looping approach would.
DISCLAIMER :- Code below may not be syntactically correct but i hope you get the picture of what I am proposing.
CREATE TYPE TableType AS TABLE
(
MetricId INT,
Descr VARCHAR(300) --or whatever length is appropriate,
EntryDE INT
);
GO
CREATE PROCEDURE [dbo].[Admin_Fill]
#RowsForUpdate TableType READONLY,
#RowsForDelete TableType READONLY
AS
BEGIN
-- Update all the Descriptions for all the rows
UPDATE
t
SET
t.Descr = u.Descr
FROM
tbTotal t
INNER JOIN #RowsForUpdate u
ON t.EntryDE = u.EntryDE AND t.MetricId = u.MetricId
-- Delete the rows to be deleted
DELETE t
FROM tbTotal t
INNER JOIN #RowsForDelete d
ON t.EntryDE = u.EntryDE AND t.MetricId = u.MetricId
END
I am reading data from csv file and adding data in database. At time of inserting data into database I want to update data with same primary key.
e.g.) I am using two Columns Bar-codes (PK) and Quantity. So, when I insert data from csv file similar barcode quantity will get added.
Can anyone help me? I am using C#.NET and SQL.
Thanks,
Rushabh Shah.
check out the merge keyword. it should do pretty much waht you're asking for.
here's a stored proc that should do it for you.
CREATE PROCEDURE dbo.InsertBarcodeData
#Barcode varchar(255),
#Quantity int
AS
BEGIN
SET NOCOUNT ON;
MERGE myTableName AS target
USING (SELECT #Barcode, #Quantity) AS source (BarCode, Quantity)
ON (target.Barcode= source.Barcode)
WHEN MATCHED THEN
UPDATE SET Quantity = source.Quantity + target.Quantity
WHEN NOT MATCHED THEN
INSERT (BarCode, Quantity)
VALUES (source.BarCode, source.Quantity)
END;
GO
create procedure InsertOrUpdateSales
(
#bar_code nvarchar(100),
#quantity int
)
as
if exists (select * from sales where bar_code = #bar_code)
update sales set quantity = quantity + #quantity where bar_code = #bar_code
else
insert into sales ( bar_code, quantity) values ( #bar_code, #quantity )
go
And
public static void InsertOrUpdateSales(string connection, string barCode, int quantity)
{
using(SqlConnection conn = new SqlConnection(connection))
{
using(SqlCommand comm = new SqlCommand("InsertOrUpdateSales", conn))
{
comm.CommandType = CommandType.StoredProcedure;
comm.Paramters.AddWithValue("#bar_code", barCode);
comm.Paramters.AddWithValue("#quantity", quantity);
comm.ExecuteNonQuery();
}
}
}
Alternatively, if you want to use the merge statement (as #Chris Lively and #nathan gonzalez mentioned) you could get really fancy and do it like this:
BULK INSERT the data from the CSV file to an empty temp table.
MERGE the temp table with the existing table.
TRUNCATE the temp table.
This might give you the best results. (For certain values of "best".)
If you can assume that there is already an existing entry for all of the bar codes in the table you could do this with a Stored procedure with two incominig parameters (#BarCodeID and #AdditionalQuantity)
UPDATE yourTable SET Quantity = Quantity + #AdditionalQuantity WHERE BarCode = #BarCodeID
You can add a Trigger to the table. When ever something is inserted in the table, you can have it run a stored procedure.
I am currently working in C#, and I need to insert a new record into one table, get the new primary key value, and then use that as a foreign key reference in inserting several more records. The Database is MS SQL Server 2003. All help is appreciated!
The way to get the identity of the inserted row is with the SCOPE_IDENTITY() function. If you're using stored procedures then this would look something like the following to return the row identity as an output parameter.
CREATE PROCEDURE dbo.MyProcedure
(
#RowId INT = NULL OUTPUT
)
AS
INSERT INTO MyTable
(
Column1
,Column2
,...
)
VALUES
(
#Param1
,#Param2
,...
);
SET #RowId = SCOPE_IDENTITY();
You can then use this value for any subsequent inserts (alternatively, if you can pass the data all into the stored procedure, then you can use it in the remainder of the procedure body).
If you're passing the SQL in dynamically then you use much the same technique, but with a single string with statement delimiters (also ; in SQL), e.g.:
var sql = "INSERT INTO MyTable (Column1, Column2, ...) VALUES (#P1, #P2, ...);" +
"SELECT SCOPE_IDENTITY();";
Then if you execute this using ExecuteScalar you'll be able to get the identity back as the scalar result and cast it to the right type. Alternatively you could build up the whole batch in one go, e.g.
var sql = "DECLARE #RowId INT;" +
"INSERT INTO MyTable (Column1, Column2, ...) VALUES (#P1, #P2, ...);" +
"SET #RowId = SCOPE_IDENTITY();" +
"INSERT INTO MyOtherTable (Column1, ...) VALUES (#P3, #P4, ...);";
This may not be exactly the right syntax, and you may need to use SET NOCOUNT ON; at the start (my mind is rusty as I rarely use dynamic SQL) but it should get you on the right track.
The best way of doing this is the use SCOPE_IDENTITY() function in TSQL. This should be executed as part of the insert i.e.
SqlCommand cmd = new SqlCommand(#"
INSERT INTO T (Name) VALUES(#Name)
SELECT SCOPE_IDENTITY() As TheId", conn);
cmd.AddParameter("#Name", SqlDbType.VarChar, 50).Value = "Test";
int tId = (int)cmd.ExecuteScalar();
Alternatively you can assign SCOPE_IDENTITY() to a variable to be used in successive statements. e.g.
DECLARE #T1 int
INSERT INTO T (Name) VALUES('Test')
SELECT #T1 = SCOPE_IDENTITY()
INSERT INTO T2 (Name, TId) VALUES('Test', #T1)
If you are just using SQL then check Duncan's answer. If however you are using LINQ then you can create the entity, save it to the DB and the ID parameter will be populated automatically.
Given a user entity and a user table it might look like this:
using(var db = new DataContext()) {
var user = new User { Name = "Jhon" };
db.Users.InsertOnSubmit(user);
db.SubmitChanges();
/* At this point the user.ID field will have the primary key from the database */
}