I've written a program in VB (although I'm equally comfortable with C#, so please answer in either language) that is used to insert a large amount of data into a table in my SQL server DB.
The way I've planned on doing it is:
To get the table's Schema from the DB using the SqlDataAdapter.FillSchema method.
Add the many rows of data to my DataTable
Use SqlBulkCopy (.WriteToServer(dtToCopy.CreateDataReader)) to load in the data
I've done this many times before and it's proven very efficient. My problem is that the table I'm writing to has an auto-incrementing primary key column, RowNum.
Now I know I could re-query my table after getting the schema and get the latest value of RowNum and simply write in the values myself into the corresponding column in my DataTable, but my fear is that the table may change between the time of my query and the time of my write, in which case, I could have un-desirable results.
I've seen posts relating to MissingSchemaAction > AddWithKey, but I'm not sure exactly how I could implement it in my case?
Would I just remove the primary key and delete the RowNum column from my DataTable?
And, if so, at what point do I set that property? It doesn't seem to be a property of the DataTable and I don't see where to add it in the SqlBulkCopy.
Thanks!!!
After a lot of searching, I found the solution myself.
Basically, I had to:
Set the PrimaryKey property of my DataTable to Nothing
Remove the RowNum column from my DataTable
Use ColumnMappings for my SqlBulkCopy so every other column is mapped and the identity is left out
Then, since it is a primary key and set as auto-increment in the DB, it gets automatically assigned the correct values at the time of load.
Hope this helps others!!!
Main Source:
SqlBulkCopy Insert with Identity Column
Related
I have created table in MS SQL 2008 with one identity column(Start Value - 1 and Increment is also 1) and 4 other columns. I am accessing this DB from C# ASP.NET. Used to push data only for the non identity column. Identity column will auto increment itself.
As of now i am manually querying the column value with the remaining for columns. But I am facing problem if all the other four column values are equal i am not getting the exact value which i am looking for
Now my query is, Is there any why in C# where I can get the value of the newly created identity column whenever new record is created.
Thanks.
You can use
SCOPE_IDENTITY()
Which will returns the primary key value of the recently inserted row
The answer to your question actually lies in SQL Server. You can run:
SELECT ##identity
after your insert to get the last inserted row identity.
http://technet.microsoft.com/en-us/library/aa933167(v=sql.80).aspx
EDIT BASED ON COMMENTS:
Consider using SCOPE_IDENTITY() as referenced here:
http://technet.microsoft.com/en-us/library/aa259185(v=sql.80).aspx
In SQL terms you can output the records back if you wish it. But how you might apply this to C# is up to you. Example:
INSERT INTO TABLE_A (SOMETHING, SOMETHINGELSE, RANDOMVAL3)
OUTPUT inserted.A_ID, inserted.SOMETHING, inserted.SOMETHINGELSE, inserted.RANDOMVAL3
SELECT 'ASD','DOSD', 123
But unless you're using merge, you can't use OUTPUT to print out any values from joining tables from an INSERT. But that's another matter entirely, I think.
Also, it's hardly good practice to bounce this data between the application and the DB all the time, so I'd look to alternatives if possible.
I have found that LINQ to Entities needs a primary key on the table in order to return correct results. Without it, I get the expected number of rows but including duplicates (and accordingly, missed rows). This problem is described here and here, and I consider it to be a bug.
In one of my tables, each row is unique but I cannot create a compound key across all fields because nullable columns cannot be used in primary keys (again, I consider this a SQL Server limitation).
So... how can I get correct results when selecting from this table using LINQ to Entities? I believe the "key" may be to create an "Entity Key" across all columns in the Visual Studio model designer but I'm not sure how to do this. Setting Entity Key = true on nullable columns throws an exception.
At one stage I gave up and added an identity int column with auto-increment enabled and used that as PK, which solved the issue, but I had to throw this out because of the volume of data being deleted/inserted all the time (it's not possible to simply reset the auto-increment counter on a schedule because not all of the rows are deleted, causing clashes).
My last resort will be to add a bigint identity column as PK, get rid of the auto-increment seed value resetter and hope it lasts "long enough" for the life of the application, but I'm not comfortable with this. (Edit: OK... it will last long enough. My main concern is performance)
I'm having such a problem: got dataset with a table(s). Say, we have already several records in a table which has a primary key (autoincrement). Basically, program works fine until I insert a new row. Even I use method AcceptChanges or Update, the new row is commited but it existing datatable object lacks the newest primary key value which I need for furhter processing. Is the only method to reload all the table from database again or there is a better way for it?
Thanks,
Ray
You should add ';select SCOPE_IDENTITY()' to the end of your insert sql statement attached to your data adapter. That will read back the last inserted value generated by autoincrement and update your dataset.
I.e, your insert sql should resemble this:
insert into sometable(column1,column2) values(#column1,#column2); select * from sometable where primarykeycolumn=scope_identity()
Your existing dataset will then contain the primary keys when dataadaper.Update(dataset) returns.
If you have a connection object you could use
int = (int)conn.executeScalar("SELECT ##IDENTITY")
Modified by what datatype your primary key is.
This has to be done directly before any other operations as a new insert will replace the stored value in ##IDENTIY
I'm writing a quick app using LINQ to SQL to populate a db with some test data and had a problem because one of the tables had no primary key as described by this bloke Can't Update because table has no primary key.
Taking the top answer I added the IsPrimaryKey attribute to an appropriate column and the app worked even though the I haven't changed the db table itself (i.e. there is still no primary key).
I expect it will be ok for my current intentions but are there any side effects which may come from having a table without a primary key seen as having one by the LINQ object?
(I can only think it might be a problem if I tried to read from a table (or populate to a table) with data where the 'primary key' column has the same value in more than one row).
When using an ORM framework, you can simulate keys and foreign keys at ORM level, thus "hiding and overriding" the database defined ones.
That said, that's a practice that I wouldn't recommend. Even if the model is more important than the database itself, the logical structure should always match. It is ok doing what you did if you're forced to work with a legacy database and you don't have the possibility to fix it (like adding the PK on the table). But try to walk the righteous path everytime you can :)
Tables without a PK = Pure Evil.
Basically if all the table updates go through the LINQ object you should be fine. If you have a DBA that decides to modify data directly though SQL then you can quickly run into issues if he duplicates a row with the same PK value.
I have a strongly typed DataTable created with the VS2005/VS2008 DataSet designer.
The table has a Primary Key column that is a guid, which gets populated by SQL server. The problem is when I want add a row (or multiple rows) to my DataTable and then call the DataAdapter.Update method (passing in the DataTable). When DataAdapter.Update is called I get a SQL exception saying that I cannot insert NULL into the primary key column.
How do I tell the designer that this is an autogenerated column and I do not want to provide a value for new rows? I just want the value generated by SQL.
Am I missing something here, or is this a limitation of the DataSet designer?
I know how achieve this using LINQ to SQL, but unfortunatley I do not have it at my disposal for this project.
Possibly one of these:
If you don't need the column in your DataSet for your app, then remove it.
If you want the column but don't care to give it a value, then change it to allow DBNull.
You can always turn off constraint enforcement (probably a bad idea): DataSet.EnforceConstraints = false
You could fill the column with a surrogate key that does not get sent to the DB.
For the first two options, if you want the convenience of letting the designer keep your structure in sync with your database, then you could remove the column or allow null programmatically, perhaps right next to a "// HACK: " comment explaining why.
You're problably using DEFAULT NEWID() on your SQL Server table definition so the problem may be that the Dataset designer doesn't see this column as auto-generated.
Maybe generating guid in your application code could be a solution? If not, then you could set default value in you Dataset but then you're probably have to also change DataAdapter Insert/Update statements so that this default value doesn't get inserted into Sql Server table.
There could also be some other solution that I'm not aware of...