Develop Reference

Select all columns except the first column for any given SQL Server table

I have this code in C#, but I need it to select all columns EXCEPT the first column of the table (the identity column), so that when I insert the data into an identical table in a different database, the destination database assigns its own identity column values:
SqlCommand commandSourceData = new SqlCommand($"SELECT * FROM dbo.{tableName};", sourceConnection);
SqlDataReader reader = commandSourceData.ExecuteReader();
Is there a way to do this?

If you want a generic solution for every column in your database you can use this kind of code
public string GetColumnsWithoutIdentity(string tableName, SqlConnection con)
{
SqlDataAdapter da = new SqlDataAdapter($"SELECT * FROM dbo.{tableName} where 1=0", con);
DataTable dt = new DataTable();
da.FillSchema(dt, SchemaType.Source);
var cols = dt.Columns.Cast<DataColumn>().Where(x => !x.AutoIncrement).Select(x => x.ColumnName);
return string.Join(",", cols);
}
Now you can use the returned string to build an Sql statement without the autoincrement column.
Notice that this code is vulnerable to Sql Injection. You should be absolutely sure that the tableName parameter used to build the first query is not typed directly by your user. Let it choose from a whitelist (readonly) of predefined tables (and also this is not 100% safe)
Another drawback is the fact that you need to hit the database two times. Once to get the schema with the info about the AutoIncrement column and one to fill the datatable after that.

Remove a column from select statement

I would like to know is there a way to remove a specific column when displaying in a datagridview. Below is the sql statemnt to retrieve
OleDbDataAdapter oda = new OleDbDataAdapter("select * from BHR_2016_FEB_CIT4114_FYP_GD", con);
DataTable dt = new DataTable();
oda.Fill(dt);
dt.Columns.Remove("Fingerprint_Template");
dataGridViewAttendanceDatabase.DataSource = dt;
The reasons is because, i have a type of format CLOB, therefore it will not display in datagridview. So i plan remove a column which stores the CLOB format. I can specify, but the problem i will be selecting table based on combo box and each table have different number of columns such as a table might have 31 column, another table might have 28 column. So how i could remove only a single column. Thanks in advanced.
There is still an error as when i state select *, it includes the column which consist of CLOB format. Therefore error exist at oda.Fill(dt);. Is there a way to select * except a columns which consist of CLOB format. This help is really appreciated.

you can just do this
dt.Columns.Remove("xyz");
Update
It seems Fill does not support CLOB data. Above solution will not work.You need to do change in SQL itself. My suggestion is to move everything in stored procedure and use meta data to extract column which you want (or exclude column which you do not want.) Here is sample code. Please note I have not tested it so you may find some minor issue but the code gives you gist of what you can do to solve your problem
C# Code
OleDbConnection oc= new OleDbConnection("[pass your connection string]");
OleDbCommand ocom = new OleDbCommand();
ocom.CommandText = "Abc"; // Abc is stored procedure
ocom.Connection = oc;
ocom.CommandType = CommandType.StoreProcedure;
ocom.Parameters.AddWithValue("#tableName","PQR") // pass your table name
ocom.Parameters.AddWithValue("#databaseName","IJK"); // pass your database name
OleDbDataAdapter oda = new OleDbDataAdapter(ocom);
DataTable dt = new DataTable();
oda.Fill(dt);
dataGridViewAttendanceDatabase.DataSource = dt;
SQL Script
Create Porocedure Abc (#tableName varchar(255),#databaseName varchar(255))
Begin
SET #sql = CONCAT('SELECT ', (SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME), 'Fingerprint_Template,', '') FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #tableName AND TABLE_SCHEMA = #databaseName), CONCAT(' FROM ',#tableName));
PREPARE stmt FROM #sql;
EXECUTE stmt;
End

Update and insert records into Oracle table using OracleDataAdapter from DataTable

a question from a newbie to c# and apologies for the lenght of it. I have the following scenario. I have a small console application that populates a datatable by connecting to an external system and then needs to update existing records and insert new ones into an oracle table. The columns in the datatable are not named the same as the oracle table columns and not in the same order. I read another post on here with a similar scenario (loading from a file into a table) and it mentioned that doing an update/insert with an OracleDataAdapter would work. A simplified datatable and oracle table are
DataTable table = new DataTable();
table.Columns.Add("Product", typeof(String));
table.Columns.Add("Price", typeof(double));
table.Columns.Add("Effective_Date", typeof(DateTime));
//sample data
table.Rows.Add("abcd", 1.011, DateTime.Today);
table.Rows.Add("efg", 1.00, DateTime.Today);
table.Rows.Add("hijk", 20, DateTime.Today);
The oracle table has the structure
ITEM VARCHAR2(20 BYTE) NOT NULL ENABLE,
EFF_DATE DATE,
VALUE NUMBER
I have tried the following code to use the datatable and an adapter to update the oracle table but I'm missing something. I'm also wondering if I'm barking up the wrong tree. The majority of examples I have seen of using a dataadapter first does a select from the table and then puts the results into a grid where a user would be able to add, update, insert, or delete records and then uses the dataadapter to update the table. In my case I'm wondering if I get it to work if all records in the datatable will be treated as an insert anyway as there is no connection between the datatable and the oracle table.
I'm using the Oracle.ManagedDataAccess.Client to connect and do the updates
public static void UpdateOrSaveItems(DataTable dt)
{
String insert_statement, update_statement, select_statement;
select_statement = "SELECT * from items";
insert_statement = "INSERT INTO items (item, eff_date, value) values (:pInsItem,:pInsEffDate,:pInsValue)";
update_statement = "UPDATE items set eff_date = :pUpdEffDate, value = :pUpdValue where item = :pUpdItem";
using (OracleConnection conn = theDatabase.ConnectToDatabase())
{
using (OracleDataAdapter oraAdapter = new OracleDataAdapter(select_statement, conn))
{
//build update/insert commands and parameters
oraAdapter.UpdateCommand = new OracleCommand(update_statement, conn);
oraAdapter.InsertCommand = new OracleCommand(insert_statement, conn);
oraAdapter.UpdateCommand.BindByName = true;
oraAdapter.InsertCommand.BindByName = true;
OracleParameter pUpdItem = new OracleParameter("pUpdItem", OracleDbType.Varchar2);
pUpdItem.SourceColumn = dt.Columns[0].ColumnName;
OracleParameter pUpdEffDate = new OracleParameter("pUpdEffDate", OracleDbType.Date);
pUpdEffDate.SourceColumn = dt.Columns[2].ColumnName;
OracleParameter pUpdValue = new OracleParameter("pUpdValue", OracleDbType.Double);
pUpdValue.SourceColumn = dt.Columns[1].ColumnName;
OracleParameter pInsItem = new OracleParameter("pInsItem", OracleDbType.Varchar2);
pUpdItem.SourceColumn = dt.Columns[0].ColumnName;
OracleParameter pInsEffDate = new OracleParameter("pInsEffDate", OracleDbType.Date);
pInsEffDate.SourceColumn = dt.Columns[2].ColumnName;
OracleParameter pInsValue = new OracleParameter("pInsValue", OracleDbType.Double);
pInsValue.SourceColumn = dt.Columns[1].ColumnName; oraAdapter.UpdateCommand.Parameters.Add(pUpdItem);
oraAdapter.UpdateCommand.Parameters.Add(pUpdEffDate);
oraAdapter.UpdateCommand.Parameters.Add(pUpdValue);
oraAdapter.InsertCommand.Parameters.Add(pInsItem);
oraAdapter.InsertCommand.Parameters.Add(pInsEffDate);
oraAdapter.InsertCommand.Parameters.Add(pInsValue);
oraAdapter.Update(dt);
}
}
}
When I run this I get an error that I cannot insert a null into column that is defined as a key. In the datatable none of them are null. I'm missing something on telling it where the data is but am unsure what it is. Also wondering if this is the right way to do this sort of thing. I wanted to avoid
loop through datatable
select to see if record is in oracle table
if in table update else insert
because the volume of records could a couple of hundred thousand and wasn't sure what the performance would be like.

Are you initializing ColumnName properties of the DataTable object that you are passing in? If not, they could be reading as null.
For instance
public static void Main()
{
Datatable myDataTable = new DataTable();
myDataTable.Columns = new Columns[3];
myDataTable.Columns[0].ColumnName = "Employees";
myDataTable.Columns[1].ColumnName = "Salary";
myDataTable.Columns[2].ColumnName = "Department";
UpdateOrSaveItems(myDataTable);
}

I found the error. I hadn't set the source column on one of my insert parameters. I had set the source column on the pUdpItem twice instead of setting it for pUdpItem and pInsItem

Importing Excel files with a large number of columns header into mysql with c#

i just was just wondering, how do i import large excel files into mysql with c#? My coding experience isn't great and i was hoping if there's anyone out there who could give me some rough idea to start on it. So far, i was able to load excel files into datagridview with the following codes:
string PathConn = " Provider=Microsoft.JET.OLEDB.4.0;Data Source=" + pathTextBox.Text + ";Extended Properties =\"Excel 8.0;HDR=Yes;\";";
OleDbConnection conn = new OleDbConnection(PathConn);
conn.Open();
OleDbDataAdapter myDataAdapter = new OleDbDataAdapter("Select * from [" + loadTextBox.Text + "$]", conn);
table = new DataTable();
myDataAdapter.Fill(table);
but after that, i don't know how i could extract the information and save it into mysql database. Assuming i have a empty scheme created before, how do i work on uploading excel files into mysql? thanks.

I think you would then need to loop over the items in the datatable and do something with them (maybe an insert statement to your DB)
like so
foreach(DataRow dr in table.Rows)
{
string s = dr[0].ToString() // this will be the first column in the datatabl as they are zero indexed
}

this is what i do in data migration scenarios from one SQL Server to another or DataFiles to SQL:
Create the new Table on the destination SQL Server (Column names, Primary Key etc.)
Load existing Data into a DataTable (Thats what you did already)
Now Query the new Table with the DataAdapter into another DataTable (Same as you did with the excel file except you now query the SQL Table.)
Load OldData from 'table' into 'newTable' using DataTable Method "Load()"
string PathConn = (MYSQL Connection String goes here)
OleDbConnection conn = new OleDbConnection(PathConn);
conn.Open();
OleDbDataAdapter myDataAdapter = new OleDbDataAdapter("Select * from [" + loadTextBox.Text + "$]", conn);
newTable = new DataTable();
myDataAdapter.Fill(newTable);
Now use the Load() Method on the new table:
newTable.Load(table.CreateDataReader(), <Specify LoadOption here>)
Matching columns will be imported into the new DataTable. (You can ensure the mapping through using Aliases in the select statements)
After Loading the existing Data into the new Table you will be able to use an DataAdapter to write the changes back to database.
Example for writing data back: ConnString - connection String for DB,
SelectStmt (can use the same as you did on the empty Table before) and provide the newTable as dtToWrite
public static void writeDataTableToServer(string ConnString, string selectStmt, DataTable dtToWrite)
{
using (OdbcConnection odbcConn = new OdbcConnection(ConnString))
{
odbcConn.Open();
using (OdbcTransaction trans = odbcConn.BeginTransaction())
{
using (OdbcDataAdapter daTmp = new OdbcDataAdapter(selectStmt, ConnString))
{
using (OdbcCommandBuilder cb = new OdbcCommandBuilder(daTmp))
{
try
{
cb.ConflictOption = ConflictOption.OverwriteChanges;
daTmp.UpdateBatchSize = 5000;
daTmp.SelectCommand.Transaction = trans;
daTmp.SelectCommand.CommandTimeout = 120;
daTmp.InsertCommand = cb.GetInsertCommand();
daTmp.InsertCommand.Transaction = trans;
daTmp.InsertCommand.CommandTimeout = 120;
daTmp.UpdateCommand = cb.GetUpdateCommand();
daTmp.UpdateCommand.Transaction = trans;
daTmp.UpdateCommand.CommandTimeout = 120;
daTmp.DeleteCommand = cb.GetDeleteCommand();
daTmp.DeleteCommand.Transaction = trans;
daTmp.DeleteCommand.CommandTimeout = 120;
daTmp.Update(dtToWrite);
trans.Commit();
}
catch (OdbcException ex)
{
trans.Rollback();
throw ex;
}
}
}
}
odbcConn.Close();
}
}
Hope this helps.
Primary Key on the newTable is necessary, otherwise you might get a CommandBuilder exception.
BR
Therak

Your halfway there, You have obtained the information from the Excel spreadsheet and have it stored in a DataTable.
The first thing you need to do before you look to import a significant amount of data into SQL is validate what you have read in from the spreadsheets.
You have a few options, one of which is do something very similar to how you read in your data and that is use a SQLAdapter to perform am INSERT into the SQL Database. All your really needing to do in this case is create a new connection and write the INSERT command.
There are many example of doing this on here.
Another option which i would use, is LINQ to CSV (http://linqtocsv.codeplex.com/).
With this you can load all of your data into class objects which makes it easier to validate each object before you perform your INSERT into SQL.
If you have limited experience then use the SQLAdapter to connect to you DB.
Good Luck

how to update datatable in c# with code?

I want move data from database to another database.
I write 2 function. function 1 : I fill table from database1 into a datatable and named this DT
in function 2 I fill table in database2 with Dt and named its dtnull
I update dtnull in database 2
function 2:
{
SqlDataAdapter sda = new SqlDataAdapter();
sda.SelectCommand = new SqlCommand();
sda.SelectCommand.Connection = objconn;
sda.SelectCommand.CommandText = "Select * from " + TableName + "";
DataTable dtnull = new DataTable();
sda.Fill(dtnull);
SqlCommandBuilder Builder = new SqlCommandBuilder();
Builder.DataAdapter = sda;
Builder.ConflictOption = ConflictOption.OverwriteChanges;
string insertCommandSql = Builder.GetInsertCommand(true).CommandText;
foreach (DataRow Row in Dt.Rows)
{
dtnull.ImportRow(Row);
}
sda.Fill(dtnull);
sda.Update(dtnull);
}

If you need to copy SQL database then just back it up and restore. Alternatively use DTS services.

If it's just a few tables I think you can
right click on the table you want in the SQL Management studio
generate a create script to your clipboard
execute it
Go back to your original table and select all the rows
copy them
go to your new table and paste
No need to make this harder than it is.

You don't really need to use an update for this. You might try out this solution, it might be the easiest way for you do this.
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopy.aspx
If you would like a LINQ solution, I could provide you with one.

There is a lot that is left unexplained. For example, do the source table and target table have the same column structure?
Can you see both database from the same SqlConnection (i.e. are they on the same machine)? If so, you can do it all in one SQL statement. Assuming you want to copy the data from table T1 in databse DB1 to table T2 in database DB2, you would write
insert DB2.dbo.T2 select * from DB1.dbo.T1
Excecute using ExecuteNonQuery.
If the databases require different SqlConnections, I would read the data from the source using a SqlDataReader and update the target row by row. I think it would be faster than using a SqlDataAdapter and DataTable since they require more structure and memory. The Update command writes the data row by row in any event.