What is the most efficient way to update a row if it exists or add it if doesn't?
I am using Linq to SQL and have read a few posts on it, but none that are current or that solve it without multiple database calls or an old framework. Currently I just insert and if there is a duplicate the statement gives an error, Violation of PRIMARY KEY constraint.
The reason i need it to be quick is that it will eventually be hitting many thousands of records.
Knowing that round-trip calls were needed in order to do this, I loaded the table's primary key values. Then before calling db.SaveChanges I added the following code:
if (checklist.Contains(tempstats))
{
db.Stats.Add(tempstats);
}
Related
I have a table in an oracle database, lets call it Task, where I'm inserting a bunch of rows from a batch process.
I have a unique constraint set up on 4 columns, one of which is nullable (locationId, shelfId, itemId, and batchId), with one of the columns being nullable (shelfId)
In the process that's parsing the CSV file's values (read from another database table), they are batched in groups of 100 and posted to an API for further parsing (into the format of the above mentioned table) and inserted for later submission to another table (in a different schema, but with the same unique constraint). The issue I'm running into is where there are duplicates based on the above constraint in the file (they are typically sequential, and I have only ever seen one additional entry in the file). After they have been parsed, they are inserted, and I'm seeing the unique constraint exception being thrown on rows that a) do not have a row in the table and b) do not meet the unique constraint. When I remove the duplicates from the initial import file I do not get any unique constraint exceptions (which... makes sense weirdly enough).
I'm using Entity Framework in .net for the Oracle database, which I wouldn't think has anything to do with this, but it may, judging by the weirdness of this issue. I'm completely stumped as to what to do, I've tried writing additional validation and looking up the records in the table before inserting them, removing them from the initial file (which works as a work around), but I'm unsure of what to do for a long-term solution.
Example Data:
LocationId ShelfId ItemID BatchId
1 NULL 00AXXFD 1
1 NULL 00AXXFD 1
1 NULL 00FFD12 1
etc...
You are getting UK error because your input data contains duplicates. When you insert all of them at once they are part of the same transaction so Oracle sees duplicates and throws exception even before you commit. After it fails the transaction rolls back so you don't see any records inserted hence no duplicates found.
The correct approach is to remove duplicates from the input data (as you are doing) before inserting.
You use Oracle to enforce UK by committing after insertion of each row.
Note - As I was saying you mayn't be committing after inserting each row. It doesn't matter if insertion happens one by one or all at once, what matters is the transaction scope. JDBC has autocommit=true/false to enable single operation commit. When it is 'true' a transaction is committed after every operation. In general it needs to be 'false' so that you can control the transaction scope
I am developing an web app using asp.net. I am making this app for compatable for both SQL and MYSQL DB.
So my concern is think that I have a set of records in a table. This table's records are referenced by other tables. So if a user try to delete a record from this table I have to check whether this record is referenced by other tables or not. If not then user can delete the record. I am using using foreign keys for many tables but others not.
So I want this scenario for every tables. So method that coming to my mind is before delete a record I have to run some select queries against those tables to check whether if records available. So is this the only approach.? Seems to its headache. you know if table is referenced by lot of tables. Can I use a flag or some thing?
Is there any better way to do this?
I think this might help you ::
SELECT
table_name, column_name
FROM
information_schema.key_column_usage
WHERE
referenced_table_name = '<table>'
and referenced_column_name = '<primary key column>'
Please check this link too:
MySQL: How to I find all tables that have foreign keys that reference particular table.column AND have values for those foreign keys?
I think it is a little overkill and not performance optimized to be selecting tables and references to check before each delete. You will be making unnecessary database calls.
Since you tag'd ASP.Net are you using ADO ? If so, or similar.
Why not make the normal delete call inside a try block and in the catch handle error message received from database something like:
try
{
}
catch(SqlExcpetion sqlEx)
{
if(sqlEx.Message.ToLower().Contains("foreign"))
{
return "your user friendly error message";
}
}
In case you are using foreign keys to constraint the references, you can act in the following order:
consider you are using database test and are trying to delete a row from emp table
1) list all the tables with their column names, that reference any column in the table we are going to remove a row from (emp in this case)
select
table_name,column_name,referenced_column_name
from
information_schema.KEY_COLUMN_USAGE
where
REFERENCED_TABLE_NAME = 'emp' and REFERENCED_table_schema = 'test';
2) for each row of the result try looking up the value of referenced_column_name from the emp row that is being removed in the corresponding table_name.column_name
I have found that LINQ to Entities needs a primary key on the table in order to return correct results. Without it, I get the expected number of rows but including duplicates (and accordingly, missed rows). This problem is described here and here, and I consider it to be a bug.
In one of my tables, each row is unique but I cannot create a compound key across all fields because nullable columns cannot be used in primary keys (again, I consider this a SQL Server limitation).
So... how can I get correct results when selecting from this table using LINQ to Entities? I believe the "key" may be to create an "Entity Key" across all columns in the Visual Studio model designer but I'm not sure how to do this. Setting Entity Key = true on nullable columns throws an exception.
At one stage I gave up and added an identity int column with auto-increment enabled and used that as PK, which solved the issue, but I had to throw this out because of the volume of data being deleted/inserted all the time (it's not possible to simply reset the auto-increment counter on a schedule because not all of the rows are deleted, causing clashes).
My last resort will be to add a bigint identity column as PK, get rid of the auto-increment seed value resetter and hope it lasts "long enough" for the life of the application, but I'm not comfortable with this. (Edit: OK... it will last long enough. My main concern is performance)
I am working on a project in C# using sql server as a data base. So the problem is there is a algo in my project which returns a single value every time; which is saved to the database (as my project requirement). If the algo repeats a value, that will also save to the database which is not required (duplication) and cause some problems. I need help to overcome the problem that a unique value saves only once when it occurs; no repetition in database. I tried to make that column a primary key, but then I found primary key violation exception.
Just use an EXISTS query either in-line or in a stored procedure
Your query procedure can check if the row already exists
Edit: Aha sorry forgot the NOT, that would be stupid :D
IF NOT EXISTS(SELECT UniqueValue FROM UniqueValuesTable WHERE UniqueValue = #NewValue)
INSERT INTO UniqueValuesTable VALUES (#NewValue)
Edit: here is a SQL fiddle to show it working
http://sqlfiddle.com/#!3/b87f9/3
As dems pointed out, the operation isn't atomic, so in a multi session situation there could be a PK violation still
Alternative is:
INSERT INTO UniqueValuesTable SELECT #NewValue WHERE NOT EXISTS (SELECT UniqueValue FROM UniqueValuesTable WHERE UniqueValue = #NewValue)
I'm writing a quick app using LINQ to SQL to populate a db with some test data and had a problem because one of the tables had no primary key as described by this bloke Can't Update because table has no primary key.
Taking the top answer I added the IsPrimaryKey attribute to an appropriate column and the app worked even though the I haven't changed the db table itself (i.e. there is still no primary key).
I expect it will be ok for my current intentions but are there any side effects which may come from having a table without a primary key seen as having one by the LINQ object?
(I can only think it might be a problem if I tried to read from a table (or populate to a table) with data where the 'primary key' column has the same value in more than one row).
When using an ORM framework, you can simulate keys and foreign keys at ORM level, thus "hiding and overriding" the database defined ones.
That said, that's a practice that I wouldn't recommend. Even if the model is more important than the database itself, the logical structure should always match. It is ok doing what you did if you're forced to work with a legacy database and you don't have the possibility to fix it (like adding the PK on the table). But try to walk the righteous path everytime you can :)
Tables without a PK = Pure Evil.
Basically if all the table updates go through the LINQ object you should be fine. If you have a DBA that decides to modify data directly though SQL then you can quickly run into issues if he duplicates a row with the same PK value.