Develop Reference

Fastest way to get the distinction/delta between a local table and the values of a database table

I have the following requirements:
I have many different database tables for example a table which shows different materialnumbers. There are many thousands rows in the database table (>100.000)
A user could import data with an excel file with thousends of rows.
The distinction/ difference between the excel file and the database-table should be added. The allready existing materialnumbers should be updated.
Is there a faster way than this to get the allready existing values in entity framework:
var allreadyExistingMaterialnumbersInDatabase = myDbContext.Materialnumbers.Where(x=>myExcelListMaterialnumbers.Contains(x.Materialnumber)).ToList();
After that step i have to do a update for all the data which is returned in the code example and then i have to insert all the difference between the list allreadyExistingMaterialNumbersInDatabase and myExcelListMaterialnumbers.
Thanks for your help.

Read from one table and insert into another - one row at a time

I am dealing with a huge database with millions of rows. I would like to run an SQL statement through C#, which selects 1.2 million rows from one database, and inserts them into another after parsing and modifying some data.
I originally wanted to do so by first running the select statement and parsing the data by iterating through the MySqlDataReader object which contains the data. This would be a memory overhead, so I have decided to select one row, parse it and insert into the other database, and then move onto the next row.
How can this be done? I have tried the SELECT....INTO syntax for a MySQL query, however this still seems to select all the data, and then inserts it after.

Use SqlBulkCopy Class to move data from one source to other
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopy%28v=vs.110%29.aspx

I am not sure if you are able to add a new column to the existing table. If you are able to add a new column, you can use the new column as a flag. It could be "TRANSFERED(boolean)".
You will select one row at a time with the condition TRANSFERED=FALSE and do the process. After that row is processed, you should update as TRANSFERED=TRUE.
Or, you must have a uniqe id column in your existing table. Create a temp table which will store the id of processed rows, that way you will know which rows are processed or not

I am not quite sure what is your error. For your case, I suggest you should use 'select top 1000 ' to get the data because insert row one by one is really slow. After that, you can use 'insert into query', it should be noted that sqlbulkcopy is just for sql server, I suggest you use the stringbuilder to make the sql query for if you use string, it will has a big overhead to concat the string.

Save XML data into database as columns in C#

I'm writing an C# application that watch for newly created xml files, and I want to insert some of the data in these XML files to a SQL database. One file is one row in the database.
What is the best and easiest way to do this?
What I have done so far is to defined a DataTable with the columns I want from the XML. After that I use ReadXML on the DataSet. This give me a DataSet with one table and one row, and the columns I want. So far it's perfect. But I can't find a good way to ONLY insert this new DataRow into my MSSQL database. I don't have any unique ID for that row yet. Would this make it easier? I don't want to map my dataset with the database, I'm only doing an insert...

You can do something like this to get the ID:
string query = "INSERT INTO Employees (EmployeeID, Name, Phone) "+
"OUTPUT INSERTED.EmployeeID "+
"VALUES (NEWID(), 'John Kris', '99-99999')";
Guid lastId = (Guid)command.ExecuteScalar();
The rest of the things you do sound ok for me.

I used LINQ to SQL to solve this. I set it up somewhat like described here:
LINQ to SQL.
On the data context I used InsertOnSubmit.

Insert data from DataTable to database table

What is the easiest and most efficient way to insert data from a DataTable into a SQL Server database table? For what it's worth, the DataTable and the database table have matching columns.
What I am doing is transferring data from one database to another (separate instances, so I will be using my application as the intermediate "receiver/sender" of data). I can easily populate a DataTable with a SqlDataAdapater.Fill() call from the source database table. But I'm trying to find the most proficient way to send that DataTable's data to the final destination database table.
Any suggestions/advice/opinions are much appreciated.
EDIT: the destination database table already exists.

You should take a look at the SqlBulkCopy class, particularly the overload of the WriteToServer method that takes a DataTable as a parameter.
If you want to be even more efficient, and you don't have the requirement to materialize the entire table into a DataSet (or, you can process the contents as you move them in a forward-only manner), then use the overload of WriteToServer that takes an IDataReader run your query using the ExecuteReader method on the SqlCommand class instead of using a SqlDataAdapter to load the entire table into memory.

How to save data from a DataGridView into database? c# winforms

I have this little application, and what it does is a user can add many deductions to a certain employee and save the record.
What I do in my coding is I pass all the values to a DataTable and loop through each rows and execute a stored procedure that inserts the particular row with the column values. This happens until there are no rows to be inserted from the DataTable.
Is there any shortcut? I mean, can I insert the whole value of datatable in one call?
I just thought that my current way of inserting data is very resource consuming because it always calls the stored procedure in the server as long as there are rows in the datatable.
Any work around or suggestions? I would be very thankful. I hope you understand me,. thanks in advance.

Please have a look at these links
http://www.a2zdotnet.com/View.aspx?Id=107
http://ask.sqlservercentral.com/questions/7282/how-to-send-the-datatable-from-cnet-to-sqlserver-2008