Develop Reference

Make Data columns from a sql adapter

Is there a way, where I can use a sql adapter and convert the sql query results into a Data Columns? I'm kinda new a datatables. I need to build this dynamically since my column names are stored into a sql table. I keep running into datarows not columns. What I have built so far:
string feedcolumns = #"select FeedColumnName from myTable where FeedProcessID = #feedprocessid";
SqlCommand columnscommand = new SqlCommand(feedcolumns, connUpd);
DataTable dt = new DataTable("datafeed");
foreach(DataColumn dc in dt.Columns)
{
dc = new
dt.Columns.Add(dc);
}

You can fill a DataTable directly from an SqlReader, no need to go column by column.
One thing I did notice was that your SQL statement had a parameter in it that was never assigned to the command object, so I added it in
string feedcolumns = "select FeedColumnName from myTable where FeedProcessID = #feedprocessid";
DataTable dt = new DataTable("datafeed");
using (SqlConnection connUpd = new SqlConnection("MyConnection")) {
using (SqlCommand columnscommand = new SqlCommand(feedcolumns, connUpd)) {
columnscommand.Parameters.AddWithValue("#feedprocessid", feedprocessid);
connUpd.Open();
var dataReader = columnscommand.ExecuteReader();
dt.Load(dataReader);
}
}

SqlBulkCopy - Add rows along with additional values to database

I am trying a code snippet that add excel data into sql database using SqlBulkCopy. The code snippet is as given below
OleDbConnection connection=null;
string FilePath="";
try
{
if (FileUpload1.HasFile)
{
FileUpload1.SaveAs(Server.MapPath("~/UploadedFolder/"+FileUpload1.FileName));
FilePath = Server.MapPath("~/UploadedFolder/"+FileUpload1.FileName);
}
string path = FilePath;
// Connection String to Excel Workbook
string excelConnectionString = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=Excel 8.0", path);
connection = new OleDbConnection();
connection.ConnectionString = excelConnectionString;
OleDbCommand command = new OleDbCommand("select * from [Sheet1$]", connection);
connection.Open();
// Create DbDataReader to Data Worksheet
DbDataReader dr = command.ExecuteReader();
// SQL Server Connection String
string sqlConnectionString = #"Data Source=sample;Initial Catalog=ExcelImport;User ID=sample;Password=sample";
// Bulk Copy to SQL Server
SqlBulkCopy bulkInsert = new SqlBulkCopy(sqlConnectionString);
bulkInsert.DestinationTableName = "Customer_Table";
bulkInsert.WriteToServer(dr);
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
finally
{
connection.Close();
Array.ForEach(Directory.GetFiles(Server.MapPath("~/UploadedFolder/")), File.Delete);
}
This add the data from excel file to sql server database table. But my requirement is that I need to add the values of excel sheet plus additionally my own values say autogenerated studentid.
So my question is how I will add new values (say studentid, batchcode etc) along with values that read from excel. And these values to be added to each row of excel data.
Example:-
excel contains following columns
CustomerID,City,Country,PostalCode
Now I need to add values to sql server by adding some new columns as
StudentID,CustomerID,BatchCode,City,Country,Email,PostalCode
How can I do it
Please help

You could do the following:
load the excel data into a data table,
Add the remaining columns to the data table,
Set new column values
SqlBulkCopy the data table into SQL Server.
Something like this:
DbDataReader dr = command.ExecuteReader();
DataTable table = new DataTable("Customers");
table.Load(dr);
table.Columns.Add("StudentId", typeof(int));
table.Columns.Add("BatchCode", typeof(string));
table.Columns.Add("Email", typeof(string));
foreach (DataRow row in table.Rows)
{
row["StudentId"] = GetStudentId(row);
row["BatchCode"] = GetBatchCode(row);
row["Email"] = GetEmail(row);
}
// SQL Server Connection String
string sqlConnectionString = #"Data Source=sample;Initial Catalog=ExcelImport;User ID=sample;Password=sample";
// Bulk Copy to SQL Server
SqlBulkCopy bulkInsert = new SqlBulkCopy(sqlConnectionString);
bulkInsert.DestinationTableName = "Customer_Table";
bulkInsert.WriteToServer(table);

Unable to read first row from Excel using SqlBulkCopy

I am using following code to update an Excel file into SQL Server. Code is working but it is not able to insert first row into table.
OleDbConnection OleDb = new OleDbConnection(ConnectionString);
OleDbCommand OleDbCmm = new OleDbCommand(Query,OleDb);
OleDbDataReader OleDbdr;
OleDb.Open();
if (OleDb.State == ConnectionState.Open)
{
OleDbdr = OleDbCmm.ExecuteReader();
SqlBulkCopy BulkCopy = new SqlBulkCopy(ConfigurationManager.ConnectionStrings["connstring"].ToString());
BulkCopy.DestinationTableName = "TempTable";
if (OleDbdr.Read())
{
BulkCopy.WriteToServer(OleDbdr);
}
}
OleDb.Close();

Even I was facing same problem, this is because I was using Read() Method like below.
while (dr.Read()) {
bulkcopy.WriteToServer(dr);
}
Solution to above problem is remove dr.Read() method and while loop
use
bulkcopy.WriteToServer(dr)
without any condition and Read() Method.

One possible reason for this may be that you've indicated in your connection string that first row contains column names (HDR=YES) thus that row is not treated as containing data.
EDIT
Another possible reason for this would be the call to OleDbDataReader.Read() method before passing the reader to SqlBulkCopy object. MSDN states:
The copy operation starts at the next available row in the reader. Most of the time, the reader was just returned by ExecuteReader or a similar call, so the next available row is the first row.
Thus, in your case you should not call OleDbdr.Read() because this advances the reader to the first row; you should let BulkCopy call Read() and it will start reading from the first row.
Your code should be:
OleDbdr = OleDbCmm.ExecuteReader();
SqlBulkCopy BulkCopy = new SqlBulkCopy(ConfigurationManager.ConnectionStrings["connstring"].ToString());
BulkCopy.DestinationTableName = "TempTable";
BulkCopy.WriteToServer(OleDbdr);

You need to set header for the Excel to sql as I have done in my
string conn = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + Your D S+ ";" +
"Extended Properties=Excel 8.0;";
OleDbConnection sSourceConnection = new OleDbConnection(conn);
using (sSourceConnection)
{
DataTable dtExcelData = new DataTable();
string[] SheetNames = GetExcelSheetNames(strFileName);
string[] preColumnHeader = new string[]{ "CarrierId", "StateId", "TerrCd", "ProgramId", "ClassId",
"PremTypeID","Limit50_100", "Limit100_100", "Limit100_200", "Limit300_300", "Limit300_600",
"Limit500_500","Limit500_1mil", "Limit1mil_1mil", "Limit1mil_2mil", "OtherParameter" };
sSourceConnection.Open();
string strQuery = string.Empty;
strQuery = "SELECT * FROM [" + SheetNames[0] + "]";
OleDbDataAdapter oleDA = new OleDbDataAdapter(strQuery, sSourceConnection);
oleDA.Fill(dtExcelData);
sSourceConnection.Close();
string[] colName = new string[dtExcelData.Columns.Count];
int i = 0;
foreach (DataColumn dc in dtExcelData.Columns)
{
colName[i] = dc.ColumnName;
i++;
}
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connStr))
{
bulkCopy.DestinationTableName = "tbl_test";
bulkCopy.WriteToServer(dtExcelData);
}
}

Why My SqlBulkCopy Is Not Working

I am using SqlBulkCopy object to write a datatable into an sql server table. However, everytime I recheck my database it remains intact with no changes. A
I have tried to do Google search to determine my problem but i am unable to resolve it.
The datatable came from an .xls file.
public static DataTable dt = new DataTable();
private void ExportToGrid(String path, String filen)
{
int idx = filen.IndexOf(".");
string tf = filen.Remove(idx, 4);
OleDbConnection MyConnection = null;
DataSet DtSet = null;
OleDbDataAdapter MyCommand = null;
MyConnection = new OleDbConnection("provider=Microsoft.Jet.OLEDB.4.0; Data Source='" + path + "';Extended Properties=Excel 8.0;");
ArrayList TblName = new ArrayList();
MyConnection.Open();
DataTable schemaTable = MyConnection.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
foreach (DataRow row in schemaTable.Rows)
{
TblName.Add(row["TABLE_NAME"]);
}
MyCommand = new System.Data.OleDb.OleDbDataAdapter("select * from [" + TblName[0].ToString() + "]", MyConnection);
DtSet = new System.Data.DataSet();
MyCommand.Fill(DtSet);
MyCommand.FillSchema(DtSet, SchemaType.Source);
DataTable dt = new DataTable();
dt = DtSet.Tables[0];
Session["dt"] = dt;
int x = dt.Rows.Count;
MyConnection.Close();
if (dt.Rows.Count > 0)
{
theGridView.DataSource = dt;
theGridView.DataBind();
}
if (System.IO.File.Exists(path))
{
System.IO.File.Delete(path);
}
}
This is my writer function
private void StartImport()
{
string servername = server;
string database = database;
string tbl = "dbo.LinkDb";
Stopwatch sw = new Stopwatch();
sw.Start();
SqlBulkCopy bulkCopy = new SqlBulkCopy("Data Source=" + servername + ";Initial Catalog=" + database + ";Integrated Security=SSPI", SqlBulkCopyOptions.TableLock);
bulkCopy.DestinationTableName = tbl;
bulkCopy.WriteToServer(dt);
sw.Stop();
lblResult.Visible = true;
lblResult.Text = (sw.ElapsedMilliseconds / 1000.00).ToString();
}
Below are the screenshot of the tables stored in my sql server. I assure you that I have been complying to Case Sensitive rules.
There was no exception thrown and average time elapsed is 0.018 - 0.020 secs
Appreciate any helps.
Thank you

Based on the code you have posted, you are writing an empty datatable to the database. Your "ExportToGrid" method fills dt, a DataTable declared locally, which loses scope outside of the method. Your write function is calling the static DataTable dt which is a new datatable.
Does dt need to be static? it seems as though this could be declared as
private DataTable dt;
then inside "ExportToGrid" instead of declaring another DataTable just instantiate the already declared dt rather than declaring a new one
dt = new DataTable();
Alternatively you could extract the DataTable straight from the GridView during the write Method:
DataTable dt = (DataTable)theGridView.DataSource;
bulkCopy.WriteToServer(dt);
This removes the need for variables outside of the scope of the method.
Lastly since you are storing your datatable within the session (I am not generally an advocate of storing large amounts of data in session variables but without knowing the specifics of your site I cannot really pass judgement), you could use the following:
DataTable dt = (DataTable)Session["dt"];
bulkCopy.WriteToServer(dt);

I dont see anything obvious compared to my use except for the fact that I explicitly map columns from the data table to the database table.
Using cn As New SqlConnection(DataAccessResource.CONNECTIONSTRING)
cn.Open()
Using copy As New SqlBulkCopy(cn)
copy.BulkCopyTimeout = 300
copy.ColumnMappings.Add(0, 0)
copy.ColumnMappings.Add(1, 1)
copy.ColumnMappings.Add(2, 2)
copy.ColumnMappings.Add(3, 3)
copy.DestinationTableName = "Tablename"
copy.WriteToServer(dataset.datatable)
End Using
End Using
Connection string (sql server 2000!) looks like
"data source=DBSERVERNAME;initial catalog=DBNAME;persist security info=True;user id=USERNAME;password=PASSWORD;packet size=4096"
I doubt the connection string is a problem assuming youve used it elsewhere.
Finally have you checked the data types for the columns in the dataset datatable match the ones in the database. In my experience oledb load from excel does not always produce output you might expect, date fields and columns with mixed text and numbers being perticular problems.

SQLite, Copy DataSet / DataTable to DataBase file

I have filled a DataSet with a Table that was created from another database file. The table is NOT in the database file which I want to be able to copy the Table to.
Now I want to save all those records (DataTable) to a newly created SQLite database file...
How can i do that?
Also I really want to avoid loops if this is possible.
The best answer is by me :) so i'll share it.This is loop but writes 100k entries in 2-3secs.
using (DbTransaction dbTrans = kaupykliuduomConn.BeginTransaction())
{
downloadas.Visible = true; //my progressbar
downloadas.Maximum = dataSet1.Tables["duomenys"].Rows.Count;
using (DbCommand cmd = kaupykliuduomConn.CreateCommand())
{
cmd.CommandText = "INSERT INTO duomenys(Barkodas, Preke, kiekis) VALUES(?,?,?)";
DbParameter Field1 = cmd.CreateParameter();
DbParameter Field2 = cmd.CreateParameter();
DbParameter Field3 = cmd.CreateParameter();
cmd.Parameters.Add(Field1);
cmd.Parameters.Add(Field2);
cmd.Parameters.Add(Field3);
while (n != dataSet1.Tables["duomenys"].Rows.Count)
{
Field1.Value = dataSet1.Tables["duomenys"].Rows[n]["Barkodas"].ToString();
Field2.Value = dataSet1.Tables["duomenys"].Rows[n]["Preke"].ToString();
Field3.Value = dataSet1.Tables["duomenys"].Rows[n]["kiekis"].ToString();
downloadas.Value = n;
n++;
cmd.ExecuteNonQuery();
}
}
dbTrans.Commit();
}
In this case dataSet1.Tables["duomenys"] is already filled with all the data i need to transfer to another database. I used loop to fill dataset too.

When you load the DataTable from the source database, set the AcceptChangesDuringFill property of the data adapter to false, so that loaded records are kept in the Added state (assuming that the source database is SQL Server)
var sqlAdapter = new SqlDataAdapter("SELECT * FROM the_table", sqlConnection);
DataTable table = new DataTable();
sqlAdapter.AcceptChangesDuringFill = false;
sqlAdapter.Fill(table);
Create the table in the SQLite database, by executing the CREATE TABLE statement directly with SQLiteCommand.ExecuteNonQuery
Create a new DataAdapter for the SQLite database connection, and use it to Update the db:
var sqliteAdapter = new SQLiteDataAdapter("SELECT * FROM the_table", sqliteConnection);
var cmdBuilder = new SQLiteCommandBuilder(sqliteAdapter);
sqliteAdapter.Update(table);
If the source and target tables have the same column names and compatible types, it should work fine...

The way to import SQL data to SQLite will take long time. When you want to import data in millions, It will take lot of time. So the shortest and easiest way to do that is just fill fetch the data from SQL database in a DataTable and insert all its rows to SQLite database.
public bool ImportDataToSQLiteDatabase(string Proc, string SQLiteDatabase, params object[] obj)
{
DataTable result = null;
SqlConnection conn = null;
SqlCommand cmd = null;
try
{
result = new DataTable();
using (conn = new SqlConnection(ConStr))
{
using (cmd = CreateCommand(Proc, CommandType.StoredProcedure, obj))
{
cmd.Connection = conn;
conn.Open();
result.Load(cmd.ExecuteReader());
}
}
using (SQLiteConnection con = new SQLiteConnection(string.Format("Data Source={0};Version=3;New=False;Compress=True;Max Pool Size=100;", SQLiteDatabase)))
{
con.Open();
using (SQLiteTransaction transaction = con.BeginTransaction())
{
foreach (DataRow row in result.Rows)
{
using (SQLiteCommand sqlitecommand = new SQLiteCommand("insert into table(fh,ch,mt,pn) values ('" + Convert.ToString(row[0]) + "','" + Convert.ToString(row[1]) + "','"
+ Convert.ToString(row[2]) + "','" + Convert.ToString(row[3]) + "')", con))
{
sqlitecommand.ExecuteNonQuery();
}
}
transaction.Commit();
new General().WriteApplicationLog("Data successfully imported.");
return true;
}
}
}
catch (Exception ex)
{
result = null;
return false;
}
finally
{
if (conn.State == ConnectionState.Open)
conn.Close();
}
}
It will take a very few time as compare to upper given answers.