I've searched through numerous threads to try to find an answer to this but any answer I've found suggests using a unique constraint on a single column, or multiple columns.
My problem is, I'm writing an application in C# with a SQL Server back end. One of the features is to allow a user to import a .CSV file into the database after a little bit of pre-processing. I need to find the quickest method to prevent the user from importing the same data more than once. The data will look something like
ID -- will be auto-generated in SQL Server (PK)
Date Time(datetime)
Machine(nchar)
...
...
...
Name(nchar)
Age(int)
I want to allow any number of the columns to be duplicate values, a long as the entire record is not.
I was thinking of creating another column in the database, obtained by hashing all of the columns together and making it unique but want sure if that was the most efficient method, or if the resulting hash would be guaranteed unique. The CSV files will only be around 60 MB, but there will be tens of thousands of them.
Any help would be appreciated.
Thanks
You should be able to resolve this by creating a unique constraint which includes all the columns.
create table #a (col1 varchar(10), col2 varchar(10))
ALTER TABLE #a
ADD CONSTRAINT UQ UNIQUE NONCLUSTERED
(col1, col2)
-- Works, duplicate entries in columns
insert into #a (col1, col2)
values ('a', 'b')
,('a', 'c')
,('b', 'c')
-- Fails, full duplicate record:
insert into #a (col1, col2)
values ('a1', 'b1')
,('a1', 'b1')
The code below can work to ensure that you don't duplicate the [Date Time], Machine, [Name] and Age columns when you insert the data.
It's important to ensure that at the time of running the code, each row of the incoming dataset has a unique ID on it. This code just fails to shift any rows where the ID gets selected because all four other values are already duplicated in the destination table.
INSERT INTO MAIN_TABLE ([Date Time],Machine,[Name],Age)
SELECT [Date Time],Machine,[Name],Age
FROM IMPORT_TABLE WHERE ID NOT IN
(
SELECT I.ID FROM IMPORT_TABLE I INNER JOIN MAIN_TABLE M
ON I.[Date Time]=M.[Date Time]
AND I.Machine=M.Machine
AND I.[Name]=M.[Name]
AND I.Age=M.Age
)
I am new to ASP.net and the concept of UDT in it. I used to work on PHP and so I am having difficulty in understanding the UDT concept.
This is the stored procedure written to insert data from input forms to the database(SQL Server).
The code is working fine and is written by senior developers in my company.
CREATE Procedure [dbo].[Save_Supplier]
#Supplier_UDT Supplier_UDT Readonly,
#UserName varchar(80)
AS
Begin
-------------Block 1------
Declare #TP Table(ID int,Suppliercode varchar(80),Suppliername varchar(80),GSTVATNumber int,Description varchar(80),Productlist varchar(80),Bankdetails varchar(80),
pymenttermdescription varchar(80),Currency int,Pendingpayement Varchar(80),pendingorders int,Active bit )
-------------Block 2------
Insert into #TP(ID ,Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active)
select ID ,Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active from #Supplier_UDT
-------------Block 3------
Update Supplier
set
Suppliercode=a.Suppliercode ,
Suppliername=a.Suppliername
,GSTVATNumber=a.GSTVATNumber
,Description =a.Description
,Productlist=a.Productlist
,Bankdetails=a.Bankdetails
,pymenttermdescription=a.pymenttermdescription
,Currency=a.Currency
,Pendingpayement=a.Pendingpayement
,pendingorders=a.pendingorders
,Active=a.Active
from #TP a inner join Supplier
on a.ID=Supplier.ID
-------------Block 4------
Insert into Supplier(Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active)
select Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active
from #TP where ID not in (select ID from Supplier) and Suppliercode!=''
Upto my understanding, Block 1 is simply declaration the structure of temporary table/variable.
In Block 2, the the user passed input data in stored in the temporary table/variable.
I am having difficulty in understanding Block 3 and Block 4
I don't understand what is the UPDATE query doing before the INSERT query?
What is the purpose of Block 3 and Block 4?
(The code is working all fine, without errors.)
[1] First thing I would like to notice about this source code isn't the usage of another table variable (#TP) but the missing of transaction management and also missing of error handling. There are at least two statements (last two: UPDATE and INSERT) under the risk of generating exceptions / errors at statement level (for example).
[2] I don't see any reason to use one more table variable (#TP), the first one being the parameter #Supplier_UDT Supplier_UDT. It'll create/increase tempdb contention and from developer point of view will create another dependency (for example: if we are going to change data type of one of those columns within dbo.Supplier table then we'll have to update also this stored procedure and definition of #TP column.
[3] Note: both table variables (#TP and #Supplier_UDT) have the same columns or (at least) a set of common columns: ID ,Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active. Isn't clear if data types, NULL-ability and constraints are the same.
[4] Block 3 & 4 seems to be an implementation of UPSERT pattern but for many rows (note: most of examples for UPSERT are using just one row). This means that for those suppliers that already exist in dbo.Supplier table (SQL schema should be mandatory) UPDATE statement will change/update following columns SupplierCode, SupplierName, ... with the latest values and new suppliers are INSERTed into dbo.Supplier table.
As Dan Guzman already mentioned within his comment (+1), instead of these two statements (UPDATE and INSERT), a single MERGE statement could be used:
MERGE dbo.Supplier WITH(HOLDLOCK) AS dst -- Destination table
USING #Supplier_UDT AS src ON dst.ID = src.ID -- Source table
WHEN MATCHED THEN
UPDATE
SET
Suppliercode = a.Suppliercode ,
Suppliername = a.Suppliername,
GSTVATNumber = a.GSTVATNumber,
Description = a.Description,
Productlist = a.Productlist,
Bankdetails = a.Bankdetails,
pymenttermdescription = a.pymenttermdescription,
Currency = a.Currency,
Pendingpayement = a.Pendingpayement,
pendingorders = a.pendingorders,
Active = a.Active
WHEN NOT MATCHED AND dst.Suppliercode != '' THEN -- Please make sure that Suppliercode refers to destination table and not to source table
INSERT (Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active)
VALUES (Suppliercode ,Suppliername,GSTVATNumber,Description ,Productlist,Bankdetails,pymenttermdescription,Currency,Pendingpayement ,pendingorders ,Active);
[5] Why I would use HOLDLOCK table hint ? See Dan Guzman's blog: http://weblogs.sqlteam.com/dang/archive/2009/01/31/UPSERT-Race-Condition-With-MERGE.aspx
[6] Also, there are some bugs regarding MERGE statement described here:
https://www.mssqltips.com/sqlservertip/3074/use-caution-with-sql-servers-merge-statement/
Some of them are more or less serious.
[7] If it ain't broke, don't fix it
I want to know is there any SQL query for asp.net,c# that can just re arrange auto increment coloumn values..
eg.
deleting 2 in the table:
sno
1
2
3
4
does:
sno
1
3
4
but i want re-arrangement:
sno
1
2
3
Note:
Don't want to to the numbering manually
query to create table is like this:
CREATE TABLE uid (sno int IDENTITY(1,1) PRIMARY KEY, qpname nvarchar(500), mob int, tm int)
Let your table be named Parent and table that will hold the backup is called Backup. They should have identical columns.
INSERT INTO dbo.Backup
SELECT * FROM dbo.Parent
Now truncate the parent table
TRUNCATE TABLE dbo.Parent
Now you can just insert the data back using the first command and just reversing the table names.
Remember that this may not work in all cases. You may have On delete cascade on and if that is the case, then you would loose all data from other tables also which are referencing the parent table. I think that you should never use this is you are using any Foriegn Key reference on this table.
Following are the queries which should run 1 after other to get this functionality done. This can be easily achieved in C# by executing a generic ExecuteNonQuery().
DELETE FROM TBL1 WHERE sno = #sno;
UPDATE TBL1
SET sno = sno -1
WHERE sno > #sno;
Im working on database synchronization in my app. It means I have 5 databases, but:
only in first database product could be added/removed/modified
this first database saving information about added/removed/modified product to table (with flag 1/2/3 as add/edit/remove and productID)
so first database generates INSERT script from SELECT, for example:
in my product_changes table (addedRemovedEdited INT, productID INT) I have information:
1, 15 (1 - flag means product with ID = 15 was added), or
2, 15 (2 - flag means product with ID = 15 was edited) etc.
Now using this information I can create script - and there is problem.
At this momment im creating scripts like:
SELECT (col1, col2, col3,...) FROM Product_Category;
string query = "INSERT INTO Table VALUES (#a,#b,#c)...";
SELECT (col1,col2,col3,...) FROM Product_price;
query += "INSERT INTO .......";
And I need to do it foreach tables which contains information about one single products. So for 10 products I'll have 10 * 12 (12 because there is ~12 tables about one product) blocks of code like INSERT INTO Table 1(....); INSERT INTO TABLE2(....).
Problem is also that, all data need to have same ID in every databases - so I'm using ##identity and put it into insert query. It has to be this way, because product with ID = 10 with name 'Keyboard' in mainDB = product with ID = 10 in DB10.
And the question - maybe some of you know any better (becouse that one is not so good) solution how can I create those scripts? Like query, which will take all information from my string[] a = {"Product", "Product_price", "Product_category"} tables and generate INSERT queries but - most important - where I can add ##identity.
#EDIT: I forgot. I found that solution: how i can generate programmatically "insert into" data script file from a database table?
Well, it does generate scripts, but with auto-incremented ID. And I need to add information in right order (as middle tables) for example:
INSERT INTO Product(.....) VALUES (...);
SET #pID = ##identity FROM Product;
INSERT INTO Price (priceID,.....) VALUES (...);
SET #prID = ##identity FROM Price;
INSERT INTO Product_price (priceID, productID,...) VALUES (#prID, #pID)
I'm trying to merge tables from different databases, ServerDB and ClientDB and save to ClientDB where the client's tables hold the master's tables records plus any records the user might add. On the other hand server tables could also be updated (new records inserted).
The database has relationships. The column in TableA I want to INSERT INTO values is a ForeignKey linking to TableB's PrimaryKey which is an auto-incremental column.
I'm saving all records from both databases in a merged dataset, and due to database design restrictions, I need to clear both tables on the client before inserting the merged tables from the dataset.
I first update TableB (the one with the PrimaryKey auto-increment column), but now the values of this column have nothing to do with the ForeignKey on TableA, so I update TableA and temporary inserting in the ForeignKey column the value of the first record of TableB's PK. Now I need to update TableA foreignKey column with the correct values from TableA PK column. Theres also a third column on each table that have the same values.
What the syntax of the sql statement should be? If I don't make much sense let me know and I'll post a better description.
It was a confussing question but i think you are talking about inserting values from table 1 to table 2 where table 1 value is equal to table 2 values the sql query for this operation is
INSERT INTO emp (empno,ename)
SELECT t2.deptno,
t2.dname
FROM dept t2
LEFT JOIN emp t1
ON t2.deptno = t1.deptno
in this query table 1 (emp) will insert 2 values into columns(empno and ename) from
table 2 (dept) and join is on (deptno) which is present in both tables.
You can further ask if this was not helpful.
Thank you all for your help. After struggling for a while with the "OledbException Operation must use an updateable query" I found out a solution in a similar topic: SQL Update woes in MS Access - Operation must use an updateable query
Thats the query that did the trick:
UPDATE DISTINCTROW PlaylistsSongs
INNER JOIN PlaylistsNames ON PlaylistNames.PlaylistName = PlaylistsSongs.PlaylistName
SET PlaylistID = PlaylistNames.ID