I am facing a problem at the moment, and I would like a quick solution for this.
I already found another answers about this issue but none had helped, I had read about INSERT INTO and INSERT ON DUPLICATE KEY UPDATE, but I think it doesn't really help me.
So the thing is, I have some Migrations enabled that Add new columns to existing tables, the thing is that the existing records on table prior to Migration get records with empty values even on existing columns (because I am changing the order of the columns on the new table migrated)
I am doing a SP that it's purpose is to fill the new Table migrated, according to Inserts or Updates.
I want to insert new records always except in the case that I found an existing record on the table with empty columns (column Id - primary key - is filled)
The "pseudo code" (mixing sql tables knowledge and c# syntax) would be something like this:
int i = 0;
foreach(Record item in importing_data_table)
{
if(i < tableMigrated.Length && tableMigrated[i].Column IS NULL)
{
UPDATE tableMigrated[i] SET Column = item.Column
}
else
{
INSERT INTO item
}
i++;
}
NOTE: Deleting all the rows from the tableMigrated before inserting is not possible, because there are foreign keys to that table. I got the following error trying that approach:
- The DELETE statement conflicted with the REFERENCE constraint "FK_blabla.Testing_testingTest.tableMigrated_Id".
NOTE2: Maybe my only option is using Cursors?
Thank you so much for your help in advance!
In your Stored Procedure, use the Merge Statement. Merge provides the functionality to INSERT based on your insert condition, otherwise UPDATE. Here is an example
MERGE dbo.FactBuyingHabits AS Target
USING (SELECT CustomerID, ProductID, PurchaseDate FROM dbo.Purchases) AS Source
ON (Target.ProductID = Source.ProductID AND Target.CustomerID = Source.CustomerID)
WHEN MATCHED THEN
UPDATE SET Target.LastPurchaseDate = Source.PurchaseDate
WHEN NOT MATCHED BY TARGET THEN
INSERT (CustomerID, ProductID, LastPurchaseDate)
VALUES (Source.CustomerID, Source.ProductID, Source.PurchaseDate)
Take a look at Technet - Inserting, Updating, and Deleting Data by Using MERGE
Related
I have a DataGrid View pulling some items from my database. What I want to achieve is to be able to edit the pack size or the bar_code fields. I am aware on how to update values in a database but how would I go about doing it if the data is the same? Meaning in many instances a bar code would have multiple pack sizes that is related to the one bar code number. Let's say I have the below screenshot. A data entry error was made and the bar_code and PackSize columns are the exact same. I want to change the first bar code to "1234." How would I achieve this? I can't say update barcode to 'textBox1.Text' where bar_code = '771313166386' because it would then change both data. How do I go about only focusing on one row of data at a time?
You can try using this query to update only the first row:
UPDATE TOP (1) my_table
SET bar_code = '1234'
WHERE bar_code = '771313166386'
You should have an auto-increment id column or a Primary key in your table.
I'd suggest you handle the logic of data duplicate manipulation at the backend rather than pulling them inside the grid and handle it there.
The following query will help you retrieve the duplicate records based on the mentioned columns. You can change it to UPDATE or DELETE as per your requirement.
-- Using cte and ranking function
;With CTE
As
(
Select
Product,
Description,
BarCode,
PackSize
Row_Number() Over(Partition By Product, BarCode, PackSize Order By Product) As RowNum
From YourTable
)
Select * From CTE
-- Where RowNum > 1;
Hope this is helpful :)
This might not help you directly in your answer. But, it is important to mention that your table design is incorrect. You should ensure the data integrity by creating a primary key in your table.
So when you need to update a product you have only one row to update.
Then you can add more tables and use foreign key references between them.
You need to uniquely represent the products. As per your sample data, I guess that there isn't any primary key on your table.
What you can do is either specify a unique constraint on columns to ensure that this type of data entry cannot be done.
If you cannot come up with list of columns to uniquely identify the rows, you can use surrogate keys by specifying Identity column and then while updating, always put a constraint where thisIdentityColumn=value
A data entry error was made and the bar_code and PackSize columns are
the exact same
I think this is the key. Essentially, the exact duplicates are unintentional, and the rows should be unique. Further it looks like bar_code + pack_size is your primary key (subject to data being entered correctly).
So, when you do an update, simply update the first row found that matches a bar_code and a pack_size. If it isn't unique, then the update should ensure that you are one step closer to unique rows in the database.
If you need a non-verbal answer, let me know.
I was given a task to insert over 1000 rows with 4 columns. The table in question does not have a PK or FK. Let's say it contains columns ID, CustomerNo, Description. The records needed to be inserted can have the same CustomerNo and Description values.
I read about importing data to a temporary table, comparing it with the real table, removing duplicates, and moving new records to the real table.
I also could have 1000 queries that check if such a record already exists and insert data if it does not. But I'm too ashamed to try that out for obvious reasons.
I'm not expecting any specific code, because I did not give any specific details. What I'm hoping for is some pseudocode or general advice for completing such tasks. I can't wait to give some upvotes!
So the idea is, you don't want to insert an entry if there's already an entry with the same ID?
If so, after you import your data into a temporary table, you can accomplish what you're looking for in the where clause of a select statement:
insert into table
select ID, CustomerNo, Description from #data_source
where (#data_source.ID not in (select table.ID from table))
I would suggest to you to load the data into a temp table or variable table. Then you can do a "Select Into" using the distinct key word which will removed the duplicated records.
you will always need to read the target table, unless you bulk load the target table into a temp table(in this point you will have two temp tables) compare both, eliminate duplicates and then insert in target table, but even this is not accurate, because you can have a new insert in the target table while you do this.
I am developing an web app using asp.net. I am making this app for compatable for both SQL and MYSQL DB.
So my concern is think that I have a set of records in a table. This table's records are referenced by other tables. So if a user try to delete a record from this table I have to check whether this record is referenced by other tables or not. If not then user can delete the record. I am using using foreign keys for many tables but others not.
So I want this scenario for every tables. So method that coming to my mind is before delete a record I have to run some select queries against those tables to check whether if records available. So is this the only approach.? Seems to its headache. you know if table is referenced by lot of tables. Can I use a flag or some thing?
Is there any better way to do this?
I think this might help you ::
SELECT
table_name, column_name
FROM
information_schema.key_column_usage
WHERE
referenced_table_name = '<table>'
and referenced_column_name = '<primary key column>'
Please check this link too:
MySQL: How to I find all tables that have foreign keys that reference particular table.column AND have values for those foreign keys?
I think it is a little overkill and not performance optimized to be selecting tables and references to check before each delete. You will be making unnecessary database calls.
Since you tag'd ASP.Net are you using ADO ? If so, or similar.
Why not make the normal delete call inside a try block and in the catch handle error message received from database something like:
try
{
}
catch(SqlExcpetion sqlEx)
{
if(sqlEx.Message.ToLower().Contains("foreign"))
{
return "your user friendly error message";
}
}
In case you are using foreign keys to constraint the references, you can act in the following order:
consider you are using database test and are trying to delete a row from emp table
1) list all the tables with their column names, that reference any column in the table we are going to remove a row from (emp in this case)
select
table_name,column_name,referenced_column_name
from
information_schema.KEY_COLUMN_USAGE
where
REFERENCED_TABLE_NAME = 'emp' and REFERENCED_table_schema = 'test';
2) for each row of the result try looking up the value of referenced_column_name from the emp row that is being removed in the corresponding table_name.column_name
I'm having such a problem: got dataset with a table(s). Say, we have already several records in a table which has a primary key (autoincrement). Basically, program works fine until I insert a new row. Even I use method AcceptChanges or Update, the new row is commited but it existing datatable object lacks the newest primary key value which I need for furhter processing. Is the only method to reload all the table from database again or there is a better way for it?
Thanks,
Ray
You should add ';select SCOPE_IDENTITY()' to the end of your insert sql statement attached to your data adapter. That will read back the last inserted value generated by autoincrement and update your dataset.
I.e, your insert sql should resemble this:
insert into sometable(column1,column2) values(#column1,#column2); select * from sometable where primarykeycolumn=scope_identity()
Your existing dataset will then contain the primary keys when dataadaper.Update(dataset) returns.
If you have a connection object you could use
int = (int)conn.executeScalar("SELECT ##IDENTITY")
Modified by what datatype your primary key is.
This has to be done directly before any other operations as a new insert will replace the stored value in ##IDENTIY
I am doing a conversion with SqlBulkCopy. I currently have an IList collection of classes which basically i can do a conversion to a DataTable for use with SqlBulkCopy.
Problem is that I can have 3 records with the same ID.
Let me explain .. here are 3 records
ID Name Address
1 Scott London
1 Mark London
1 Manchester
Basically i need to insert them sequentially .. hence i insert record 1 if it doesn't exist, then the next record if it exists i need to update the record rather than insert a new 1 (notice the id is still 1) so in the case of the second record i replace both columns Name And Address on ID 1.
Finally on the 3rd record you notice that Name doesn't exist but its ID 1 and has an address of manchester so i need to update the record but NOT CHANGING Name but updating Manchester.. hence the 3rd record would make the id1 =
ID Name Address
1 Mark Manchester
Any ideas how i can do this? i am at a loss.
Thanks.
EDIT
Ok a little update. I will manage and merge my records before using SQLbulkCopy. Is it possible to get a list of what succeeded and what failed... or is it a case of ALL or nothing? I presume there is no other alternative to SQLbulkCopy but to do updates?
it would be ideal to be able to Insert everything and the ones that failed are inserted into a temp table ... hence i only need to worry about correcting the ones in my failed table as the others i know are all OK
Since you need to process that data into a DataTable anyway (unless you are writing a custom IDataReader), you should merge the records before giving them to SqlBulkCopy; for example (in pseudo code):
/* create empty data-table */
foreach(row in list) {
var row = /* try to get exsiting row from data-table based on id */
if(row == null) { row = /* create and append row to data-table */ }
else { merge non-trivial properties into existing row */
}
then pass the DataTable to SqlBulkCopy once you have the desired data.
Re the edit; in that scenario, I would upload to a staging table (just a regular table that has a schema like the uploaded data, but typically no foreign keys etc), then use regular TSQL to move the data into the transactional tables. In addition to full TSQL support this also allows better logging of operations. In particular, perhaps look at the OUTPUT clause of INSERT which can help complex bulk operations.
You can't do updates with bulk copy (bulk insert), only insert. Hence the name.
You need to fix the data before you insert them. If this means you have updates to pre-existing rows, you can't insert those as that will generate the key conflict.
You can either bulk insert into a temporary table, and run the appropriate insert or update statements, only insert the new rows and issue update statements for the rest, or delete the pre-existing rows after fetching them and fixing the data before reinserting.
But there's no way to persuade bulk copy to update an existing row.