I am using a DataTable and SqlBulkCopy to insert data into a SQL Server database table.
I have managed to bulk insert the rows using the following settings but only using negative values as otherwise I get conflicts as the value is not unique:
DataTable table = new DataTable();
DataColumn column = new DataColumn();
column.DataType = System.Type.GetType("System.Int32");
column.AutoIncrement = true;
column.AutoIncrementSeed = 0;
column.AutoIncrementStep = -1;
table.Columns.Add(column);
table.Columns.Add(DB_Base.DBTable_Start, typeof(DateTime));
I have a loop that generates rows to be inserted like this:
table.Rows.Add(null,tsoOptions.start);
I then set the table and connection and write the data using the WriteToServer() method. This is all working fine and the rows appear but with negative autoincrement primary keys.
How do I modify this so that it will append the rows with a positive value which continues after the last (MAX) value without reading the max value in a separate query?
Assuming your table in the database is created properly with the column auto-increment (i.e. IDENTITY) turned on, don't duplicate this functionality in your code. Just send the records to the database with the null value for that column and the database will do its job. Comment out these lines and try:
//column.AutoIncrement = true;
//column.AutoIncrementSeed = 0;
//column.AutoIncrementStep = -1;
UPDATE
Actually the best way to do it is by not mapping the identity column at all, so comment out all these lines:
//DataColumn column = new DataColumn();
//column.DataType = System.Type.GetType("System.Int32");
//column.AutoIncrement = true;
//column.AutoIncrementSeed = 0;
//column.AutoIncrementStep = -1;
//table.Columns.Add(column);
//table.Columns.Add(DB_Base.DBTable_Start, typeof(DateTime));
And make sure you're not using SqlBulkCopyOptions.KeepIdentity (check this).
Do you need to setup this in the DataTable object? Usually I'd go on SSMS (Sql Server Management Studio) and set up the primary key as Identity and Increment to one.
When I save in the database, I just don't send anything for this column:
1) Right-click on the table in SSMS tree on the left, choose design
2) Click on your primary-key column and have a look on Column Properties section, you should set up like this:
That should do the job!
UPDATE: Like the other guy here suggested, comment out the lines of your code that you are trying to set up seed and identity and do this in SSMS instead.
Related
I read upon a document on how to retrieve the last record after inserted. However when I do, nothing gets filled on the datagridview. This is what I have.
static String connectionString = #"Data Source = (LocalDB)\MSSQLLocalDB;AttachDbFilename= C:\Users\home\Documents\C# Programs\program\Database.mdf ;Integrated Security = True";
private void displayLastInsertedRecord()
{
using (SqlDataAdapter dataAdapter = new SqlDataAdapter("select ##identity", connectionString))
{
DataTable table = new DataTable(connectionString);
dataAdapter.Fill(table);
employeeDataGridView.DataSource = table;
}
}
I checked to see if in fact the data is being stored into the database table and it actually is. It just not display factors such as their name, city, address, etc etc.Cannot figure out what I am doing incorrectly.
Your query is returning a single row, single column table, which contains the Primary Key value of the last inserted data row, in any table in your database, provided that such primary key is an IDENTITY column. Since you want the actual data, change your query from:
SELECT ##IDENTITY
to:
SELECT #LastID = ##IDENTITY; SELECT * FROM [YourTableName] WHERE [YourPrimaryKeyColumn] = #LastID
where [YourTableName] is the table you're interested in getting the inserted row from (must match last insert command, otherwise the returned dataset will either contain no rows, or have an accidentally matching record - basically, garbage data), and [YourPrimaryKeyColumn] is the name of the primary key column in the aforementioned table.
Also, consider using SCOPE_IDENTITY or IDENT_CURRENT instead.
I am using SQL Bulk copy to read data form Excel to SQL DB. In the Database, I have two tables into which I need to insert this data from Excel. Table A and Table B which uses the ID(primary Key IDENTITY) from Table A to insert corresponding row records into Table B.
I am able to insert into one table (Table A) using the following Code.
using (SqlConnection connection = new SqlConnection(strConnection)) {
connection.Open();
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection)) {
bulkCopy.DestinationTableName = "dbo.[EMPLOYEEINFO]";
try {
// Write from the source to the destination.
SqlBulkCopyColumnMapping NameMap = new SqlBulkCopyColumnMapping(data.Columns[0].ColumnName, "EmployeeName");
SqlBulkCopyColumnMapping GMap = new SqlBulkCopyColumnMapping(data.Columns[1].ColumnName, "Gender");
SqlBulkCopyColumnMapping CMap = new SqlBulkCopyColumnMapping(data.Columns[2].ColumnName, "City");
SqlBulkCopyColumnMapping AMap = new SqlBulkCopyColumnMapping(data.Columns[3].ColumnName, "HomeAddress");
bulkCopy.ColumnMappings.Add(NameMap);
bulkCopy.ColumnMappings.Add(GMap);
bulkCopy.ColumnMappings.Add(CMap);
bulkCopy.ColumnMappings.Add(AMap);
bulkCopy.WriteToServer(data);
}
catch (Exception ex) {
Console.WriteLine(ex.Message);
}
}
}
But then I am not sure how to extend it for two tables which are bound by Foreign Key relationship.Especially, Table B uses the Identity value from Table A Any example would be great. I googled it and none of the threads on SO couldn't give a Working example.
AFAIK bulk copy can only be used to upload into a single table. In order to achieve a bulk upload into two tables, you will therefore need two bulk uploads. Your problem comes from using a foreign key which is an identity. You can work around this, however. I am pretty sure that bulk copy uploads sequentially, which means that if you upload 1,000 records and the last record gets an ID of 10,197, then the ID of the first record is 9,198! So my recommendation would be to upload your first table, check the max id after the upload, deduct the number of records and work from there!
Of course in a high use database, someone might insert after you, so you would need to get the top id by selecting the record which matches your last one by other details (assuming a combination of (upto) all fields would be guaranteed to be unique). Only you know if this is likely to be a problem.
The alternative is not to use an identity column in the first place, but I presume you have no control over the design? In my younger days, I made the mistake of using identities, I never do now. They always find a way of coming back to bite!
For example to add the second data:
DataTable secondTable = new DataTable("SecondTable");
secondTable.Columns.Add("ForeignKey", typeof(int));
secondTable.Columns.Add("DataField", typeof(yourDataType));
...
Add data to secondTable.
(Depends on format of second data)
int cnt = 0;
foreach (var d in mySecondData)
{
DataRow newRow = secondTable.NewRow();
newRow["ForeignKey"] = cnt;
newRow["DataField"] = d.YourData;
secondTable.Rows.Add(newRow);
}
Then after you found out the starting identity (int startID).
for (int i = 0; i < secondTable.Rows.Count; i++)
{
secondTable["ForeignKey"] = secondTable["ForeignKey"] + startID;
}
Finally:
bulkCopy.DestinationTableName = "YourSecondTable";
bulkCopy.WriteToServer(secondTable);
I have come across a problem in using the DataAdapter, which I hope someone can help with. Basically I am creating a system, which is as follows:
Data is read in from a data source (MS-Access, SQL Server or Excel), converted to data tables and inserted into a local SQL Server database, using DataAdapters. This bit works fine. The SQL server table has a PK, which is an identity field with auto increment set to on.
Subsequent data loads read in the data from the source and compare it to what we already have. If the record is missing then it is added (this works fine). If the record is different then it needs to be updated (this doesn't work).
When doing the differential data load I create a data table which reads in the schema from the destination table (SQL server) and ensures it has the same columns etc.
The PK in the destination table is column 0, so when a record is inserted all of the values from column 1 onwards are set (as mentioned this works perfectly.). I don't change the row status for items I am adding. The PK in the data table is set correctly and I can confirm this.
When updating data I set column 0 (the PK column) to be the value of the record I am updating and set all of the columns to be the same as the source data.
For updated records I call AcceptChanges and SetModified on the row to ensure (I thought) that the application calls the correct method.
The DataAdapter is set with SelectCommand and UpdateCommand using the command builder.
When I run, I have traced it using SQL profiler and can see that the insert command is being ran correctly, but the update command isn't being ran at all, which is the crux of the problem. For reference an insert table will look something like the following
PK Value1 Value 2 Row State
== ====== ======= =========
124 Test1 Test 2 Added
123 Test3 Test4 Updated
Couple of things to be aware of....
I have tested this by loading the row to be changed into the datatable, changing some column fields and running update and this works. However, this is impractical for my solution because the data is HUGE >1Gb so I can't simply load it into a datatable without taking a huge performance hit. What I am doing is creating the data table with a max of 500 rows and the running the Update. Testing during the initial data load showed this to be the most efficient in terms of memory useage and performance. The data table is cleared after each batch is ran.
Anyone any ideas on where I am going wrong here?
Thanks in advance
Andrew
==========Update==============
Following is the code to create the insert/update rows
private static void AddNewRecordToDataTable(DbDataReader pReader, ref DataTable pUpdateDataTable)
{
// create a new row in the table
DataRow pUpdateRow = pUpdateDataTable.NewRow();
// loop through each item in the data reader - setting all the columns apart from the PK
for (int addCount = 0; addCount < pReader.FieldCount; addCount++)
{
pUpdateRow[addCount + 1] = pReader[addCount];
}
// add the row to the update table
pUpdateDataTable.Rows.Add(pUpdateRow);
}
private static void AddUpdateRecordToDataTable(DbDataReader pReader, int pKeyValue,
ref DataTable pUpdateDataTable)
{
DataRow pUpdateRow = pUpdateDataTable.NewRow();
// set the first column (PK) to the value passed in
pUpdateRow[0] = pKeyValue;
// loop for each row apart from the PK row
for (int addCount = 0; addCount < pReader.FieldCount; addCount++)
{
pUpdateRow[addCount + 1] = pReader[addCount];
}
// add the row to the table and then update it
pUpdateDataTable.Rows.Add(pUpdateRow);
pUpdateRow.AcceptChanges();
pUpdateRow.SetModified();
}
The following code is used to actually do the update:
updateAdapter.Fill(UpdateTable);
updateAdapter.Update(UpdateTable);
UpdateTable.AcceptChanges();
The following is used to create the data table to ensure it has the same fields/data types as the source data
private static DataTable CreateDataTable(DbDataReader pReader)
{
DataTable schemaTable = pReader.GetSchemaTable();
DataTable resultTable = new DataTable(<tableName>); // edited out personal info
// loop for each row in the schema table
try
{
foreach (DataRow dataRow in schemaTable.Rows)
{
// create a new DataColumn object and set values depending
// on the current DataRows values
DataColumn dataColumn = new DataColumn();
dataColumn.ColumnName = dataRow["ColumnName"].ToString();
dataColumn.DataType = Type.GetType(dataRow["DataType"].ToString());
dataColumn.ReadOnly = (bool)dataRow["IsReadOnly"];
dataColumn.AutoIncrement = (bool)dataRow["IsAutoIncrement"];
dataColumn.Unique = (bool)dataRow["IsUnique"];
resultTable.Columns.Add(dataColumn);
}
}
catch (Exception ex)
{
message = "Unable to create data table " + ex.Message;
throw new Exception(message, ex);
}
return resultTable;
}
In case anyone is interested I did manage to get around the problem, but never managed to get the data adapter to work. Basically what I did was as follows:
Create a list of objects with an index and a list of field values as members
Read in the rows that have changed and store the values from the source data (i.e. the values that will overwrite the current ones in the object). In addition I create a comma separated list of the indexes
When I am finished I use the comma separated list in a sql IN statement to return the rows and load them into my data adapter
For each one I run a LINQ query against the index and extract the new values, updating the data set. This sets the row status to modified
I then run the update and the rows are updated correctly.
This isn't the quickest or neatest solution, but it does work and allows me to run the changes in batches.
Thanks
Andrew
I have a non-typed dataset filled with data from user input (no database). There's no primary key column (my data had no need for primary key so far)! Is there any way to avoid "brute force" if i want to check if new row user is trying to insert already exists in my DataTable? How should i perform that check?
You can manually create unique constraints for your DataTable:
DataTable custTable = custDS.Tables["Customers"];
UniqueConstraint custUnique = new UniqueConstraint(new DataColumn[]
{custTable.Columns["CustomerID"],
custTable.Columns["CompanyName"]});
custDS.Tables["Customers"].Constraints.Add(custUnique);
For this example, you would get an exception (of type ConstraintException) if you tried to add a row to the table where the CustomerID and CompanyName were duplicates of another row with the same CustomerID and CompanyName.
I would just let the DataTable check these things for you internally - no point reinventing the wheel. As to how it does it (whether it is efficient or not), will have to be an exercise for you.
What you can do is use a DataView. Dataview allow you to use a where clause with the DataView's data.
Check it that way.
To check for any duplicates try
if (table.Rows.Contain(PriKeyTypeValue)) /*See if a Primary Key Value is in
the table already */
continue;
else
table.Row.Add(value1, value2, value3);
If you want to be able to insert duplicate rows but do not want to have an exception thrown set-up your primary key as a unique self-incrementing int then you can insert as many duplicates as you feel like without having to check to see if the table contains that value.you can set primary key value like the below....
DataTable table = new DataTable();
table.Columns.Add("Column", typeof(int));
DataColumn column = table.Columns["Column"];
column.Unique = true;
column.AutoIncrement = true;
column.AutoIncrementStep = 1; //change these to whatever works for you
column.AutoIncrementSeed = 1;
table.PrimaryKey = new DataColumn[] { column };
Much, much easier way:
datatable.Columns.Contais("ColumnName")
I am having an datatable which is already populated. Now i want to add few rows to that datatable ,but some of rows might already exist in the datatable.I know its unneccesary but tht is the requirement.
I tried couple of things and got "the row already exist in this table : & this row belongs to some other table" .I also tried importRow ,but i guess it avoid the duplicates by dafault.
Is there any way to do that .If the datatable has 7 rows and i want to add 3 more rows whether its already exist or not. My goal is to send 10 rows to the calling function .
Or is there any other approach altogether?
UPDATE
Using
rowsToAdd.CopyToDataTable(dsCount.Tables[2], LoadOption.PreserveChanges); works but I'm not sure it's the proper way.
You can add new rows in DataTable using code below
DataRow newRow = dataTable.NewRow();
dataTable.Rows.Add(newRow);
To check for any duplicates try
if (table.Rows.Contain(PriKeyTypeValue)) /*See if a Primary Key Value is in
the table already */
continue;
else
table.Row.Add(value1, value2, value3);
If you want to be able to insert duplicate rows but do not want to have an exception thrown set-up your primary key as a unique self-incrementing int then you can insert as many duplicates as you feel like without having to check to see if the table contains that value. Just make sure that it would suffice to have duplicates. There are plenty of examples of setting a primary key just search for it (msdn has at least one). Here is an example:
DataTable table = new DataTable();
table.Columns.Add("Column", typeof(int));
DataColumn column = table.Columns["Column"];
column.Unique = true;
column.AutoIncrement = true;
column.AutoIncrementStep = 1; //change these to whatever works for you
column.AutoIncrementSeed = 1;
table.PrimaryKey = new DataColumn[] { column };
Create a new row using the NewRow() function.
var dataTable = new DataTable();
var dataRow = dataTable.NewRow();
Then add your new row to your datatable
dataTable.Rows.Add(dataRow)
I believe you can use the NewRow() function of the DataTable if you're simply appending a row to the table.
DataTable table = new DataTable();
DataRow row = table.NewRow();
table.Rows.Add(row);
Will that not suffice?
The most straightforward method. After spending a few hours trying everything else.
DataRow dr = ds.Tables[0].NewRow();
dr["ColumnName1"] = "columnvalue"; //string
dr["ColumnName2"] = 123 //int
ds.Tables[0].Rows.Add(dr);