I have filled a DataSet with a Table that was created from another database file. The table is NOT in the database file which I want to be able to copy the Table to.
Now I want to save all those records (DataTable) to a newly created SQLite database file...
How can i do that?
Also I really want to avoid loops if this is possible.
The best answer is by me :) so i'll share it.This is loop but writes 100k entries in 2-3secs.
using (DbTransaction dbTrans = kaupykliuduomConn.BeginTransaction())
{
downloadas.Visible = true; //my progressbar
downloadas.Maximum = dataSet1.Tables["duomenys"].Rows.Count;
using (DbCommand cmd = kaupykliuduomConn.CreateCommand())
{
cmd.CommandText = "INSERT INTO duomenys(Barkodas, Preke, kiekis) VALUES(?,?,?)";
DbParameter Field1 = cmd.CreateParameter();
DbParameter Field2 = cmd.CreateParameter();
DbParameter Field3 = cmd.CreateParameter();
cmd.Parameters.Add(Field1);
cmd.Parameters.Add(Field2);
cmd.Parameters.Add(Field3);
while (n != dataSet1.Tables["duomenys"].Rows.Count)
{
Field1.Value = dataSet1.Tables["duomenys"].Rows[n]["Barkodas"].ToString();
Field2.Value = dataSet1.Tables["duomenys"].Rows[n]["Preke"].ToString();
Field3.Value = dataSet1.Tables["duomenys"].Rows[n]["kiekis"].ToString();
downloadas.Value = n;
n++;
cmd.ExecuteNonQuery();
}
}
dbTrans.Commit();
}
In this case dataSet1.Tables["duomenys"] is already filled with all the data i need to transfer to another database. I used loop to fill dataset too.
When you load the DataTable from the source database, set the AcceptChangesDuringFill property of the data adapter to false, so that loaded records are kept in the Added state (assuming that the source database is SQL Server)
var sqlAdapter = new SqlDataAdapter("SELECT * FROM the_table", sqlConnection);
DataTable table = new DataTable();
sqlAdapter.AcceptChangesDuringFill = false;
sqlAdapter.Fill(table);
Create the table in the SQLite database, by executing the CREATE TABLE statement directly with SQLiteCommand.ExecuteNonQuery
Create a new DataAdapter for the SQLite database connection, and use it to Update the db:
var sqliteAdapter = new SQLiteDataAdapter("SELECT * FROM the_table", sqliteConnection);
var cmdBuilder = new SQLiteCommandBuilder(sqliteAdapter);
sqliteAdapter.Update(table);
If the source and target tables have the same column names and compatible types, it should work fine...
The way to import SQL data to SQLite will take long time. When you want to import data in millions, It will take lot of time. So the shortest and easiest way to do that is just fill fetch the data from SQL database in a DataTable and insert all its rows to SQLite database.
public bool ImportDataToSQLiteDatabase(string Proc, string SQLiteDatabase, params object[] obj)
{
DataTable result = null;
SqlConnection conn = null;
SqlCommand cmd = null;
try
{
result = new DataTable();
using (conn = new SqlConnection(ConStr))
{
using (cmd = CreateCommand(Proc, CommandType.StoredProcedure, obj))
{
cmd.Connection = conn;
conn.Open();
result.Load(cmd.ExecuteReader());
}
}
using (SQLiteConnection con = new SQLiteConnection(string.Format("Data Source={0};Version=3;New=False;Compress=True;Max Pool Size=100;", SQLiteDatabase)))
{
con.Open();
using (SQLiteTransaction transaction = con.BeginTransaction())
{
foreach (DataRow row in result.Rows)
{
using (SQLiteCommand sqlitecommand = new SQLiteCommand("insert into table(fh,ch,mt,pn) values ('" + Convert.ToString(row[0]) + "','" + Convert.ToString(row[1]) + "','"
+ Convert.ToString(row[2]) + "','" + Convert.ToString(row[3]) + "')", con))
{
sqlitecommand.ExecuteNonQuery();
}
}
transaction.Commit();
new General().WriteApplicationLog("Data successfully imported.");
return true;
}
}
}
catch (Exception ex)
{
result = null;
return false;
}
finally
{
if (conn.State == ConnectionState.Open)
conn.Close();
}
}
It will take a very few time as compare to upper given answers.
Related
I have to update multiple records in SQL Server table from C#. Below are the steps I have to follow and below is the code.
The code is working but the process is taking much longer than expected.
I need a quick way to update 10000 records, not sure whether Bulk Copy would work for Update.
I have seen the other answers which has Bulk insert to temp and then update..But that update has a single statement and here I need to update the records in DB based on Excel data and for this I have to loop each excel record.So how can I achieve faster update.
1) Read the Excel Data and copied the data into a data table
string strDirectory = string. Empty;
strDirectory = System.IO.Directory.GetCurrentDirectory() + "\\" + "Filename.xlsx";
string Connection String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source = " + strDirectory + "; Extended Properties = \"Excel 12.0;HDR=YES;IMEX=1\"";
using (OleDbConnection conn = new OleDbConnection(Connection String))
{
conn.Open();
DataTable schemaTable = conn.GetOleDbSchemaTableOleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
DataRow schemaRow = schemaTable. Rows[0];
string sheet = schemaRow["TABLE_NAME"].ToString();
string query = "SELECT * FROM [" + sheet + "]";
OleDbDataAdapter daexcel = new OleDbDataAdapter(query, conn);
daexcel.Fill(dt);
conn.Close();
}
2) Doing some manipulations on datatable data before updating into table.
string strsqlst = string. Empty;
using (SqlConnection sqlConn = new SqlConnection(Connectionstring))
{
sqlConn.Open();
SqlCommand cmd;
StringBuilder sb = new StringBuilder();
sb.AppendLine("DataTable content:");
foreach (DataRow row in dt.Rows)
{
if (row.ItemArray[0].ToString() == "")
break;
strsqlst = "Update table Set col1= " + row.ItemArray[4].ToString() + " ,col2= " + row.ItemArray[5].ToString() + " where <Condition>'";
cmd = new SqlCommand(strsqlst, sqlConn);
cmd.CommandType = CommandType.Text;
cmd.ExecuteNonQuery();
}
sqlConn.Close();
}
The SqlCommand can be a whole SQL batch and is not limited to a single statement. So you can create a single large batch with 10,000 UPDATE statements, or divide it into for example 20 batches of 500 each.
In other words, you can create a single command with CommandText like this:
UPDATE [T] SET Col1='Value1', Col2='Value2' WHERE [Id] = 1;
...
UPDATE [T] SET Col1='Value999', Col2='Value1000' WHERE [Id] = 500;
That said, you should use parameters for all data values (to ensure SQL injection is not possible).
If you want to handle any errors (updates failing due to invalid data) you will need something a bit more sophisticated.
I want to upload Excel data into SQL Server with uploaded by (i.e which pick from session method)
I have created table name, country, address, uploaded by.
Normally the upload is working fine but I want to add the uploaded by value while uploading the excel bulk upload
I don't know how to insert the uploaded by value in bulk upload some one help me
This is my code:
try
{
myExcelConn.Open();
// GET DATA FROM EXCEL SHEET.
OleDbCommand objOleDB = new OleDbCommand("SELECT * FROM [Sheet1$]", myExcelConn);
// READ THE DATA EXTRACTED FROM THE EXCEL FILE.
OleDbDataReader objBulkReader = null;
objBulkReader = objOleDB.ExecuteReader();
// SET THE CONNECTION STRING.
string sCon = sqlconn;
using (SqlConnection con = new SqlConnection(sCon))
{
con.Open();
//SqlConnection p = new SqlConnection(#"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=C:\Users\jetback\source\repos\Projectstudent\webdown_up_load\App_Data\upload.mdf;Integrated Security=True");
// FINALLY, LOAD DATA INTO THE DATABASE TABLE.
oSqlBulk = new SqlBulkCopy(con);
oSqlBulk.DestinationTableName = "Table3"; // TABLE NAME.
oSqlBulk.ColumnMappings.Add("name", "name");
oSqlBulk.ColumnMappings.Add("address", "address");
oSqlBulk.ColumnMappings.Add("country", "country");
oSqlBulk.WriteToServer(objBulkReader);
}
lblConfirm.Text = "DATA IMPORTED SUCCESSFULLY.";
lblConfirm.Attributes.Add("style", "color:green");
}
catch (Exception ex)
{
lblConfirm.Text = ex.Message;
lblConfirm.Attributes.Add("style", "color:red");
}
finally
{
// CLEAR.
oSqlBulk.Close();
oSqlBulk = null;
myExcelConn.Close();
myExcelConn = null;
}
It's better to use DataTable if you want to add more col when there's already data in yr dataSet:
var results = new DataTable();
using(var myExcelConn = new OleDbConnection(excCnnStr))
{
using (var cmd = new OleDbCommand("SELECT * FROM [Sheet1$]", myExcelConn))
{
myExcelConn.Open();
var adapter = new OleDbDataAdapter(cmd);
adapter.Fill(results);
}
}
//add new col
results.Columns.Add("uploadedBy", typeof(System.Int32));
foreach (DataRow row in results.Rows)
{
row["uploadedBy"] = loggedInUserId; // set uploader
}
using (var con = new SqlConnection(sCon))
{
con.Open();
using (var oSqlBulk = new SqlBulkCopy(con))
{
oSqlBulk.DestinationTableName = "Table3";
oSqlBulk.ColumnMappings.Add("name", "name");
oSqlBulk.ColumnMappings.Add("address", "address");
oSqlBulk.ColumnMappings.Add("country", "country");
oSqlBulk.ColumnMappings.Add("uploadedBy", "uploadedBy");
oSqlBulk.WriteToServer(results);
}
}
You will have to get from Excel and then insert into SQL.
//After objBulkReader = objOleDB.ExecuteReader();
if (objBulkReader.HasRows)
{
while (objBulkReader.Read())
{ string newname = objBulkReader[0].ToString();
string address= objBulkReader[1].ToString();
string country= objBulkReader[2].ToString();
string uploadedby=Session["uploader"].ToString();
//insert these into SQL db by SQL connection
}
}
The 0,1 and 2 in objBulkReader are column positions in Excel.
I am trying to bulk copy from one table to another by mapping the column names as the source and destination may not have same columns always.
Source can have 8 columns and destination can have 10 .. I need to map the columns and bulk copy.
Tried the below code..didn't work..getting Error: The given ColumnName 'moduleid' does not match up with any column in data source.
Source: existingtablecolumnsPresent has [collection time],[logtime],[moduleid],[node],[reason],[time],[timestamp],[usecaseid]
Destination: dataTable.Columns has [Node],[Time],[Reason],[Moduleid],[Usecaseid]
Please advise
public static void BatchBulkCopy(DataTable dataTable, string DestinationTbl, List<string> columnMapping,string filename)
{
var program = new Program();
// Get the DataTable
DataTable dtInsertRows = dataTable;
using (SqlBulkCopy sbc = new SqlBulkCopy(program.connectionStr.ToString()))
{
try {
sbc.DestinationTableName = DestinationTbl.ToLower();
string sourceTableQuery = "Select top 1 * from " + "[" + dataTable.TableName + "]";
DataTable dtSource = SqlHelper.ExecuteDataset(program.connectionStr.ToString(), CommandType.Text, sourceTableQuery).Tables[0];
for (int i = 0; i < dataTable.Columns.Count; i++)
{ //check if destination Column Exists in Source table
if (dtSource.Columns.Cast<DataColumn>().Select(a => "[" + a.ColumnName.ToLower() + "]").Contains(dataTable.Columns[i].ToString().ToLower()))//contain method is not case sensitive
{
List<string> existingtablecolumnsPresent = dtSource.Columns.Cast<DataColumn>().Select(a => "[" + a.ColumnName.ToLower() + "]").Distinct().OrderBy(t => t).ToList();
int sourceColumnIndex = existingtablecolumnsPresent.IndexOf(dataTable.Columns[i].ToString().ToLower());//Once column matched get its index
sbc.ColumnMappings.Add(dtSource.Columns[sourceColumnIndex].ToString(), dtSource.Columns[sourceColumnIndex].ToString());//give coluns name of source table rather then destination table so that it would avoid case sensitivity
}
}
sbc.WriteToServer(dtInsertRows);
sbc.Close();
}
catch (Exception ex)
{
Log.WriteLog("BatchBulkCopy" + " - " + filename, dataTable.TableName, ex.Message.ToString());
// To move a file or folder to a new location:
//if (File.Exists(program.sourceFile + filename))
// System.IO.File.Move(program.sourceFile + filename, program.failedfiles + filename);
}
As requested (create a DataTable with the columns you want to insert in them- leave the others out. Make sure any columns you leave out are marked in the table for NULL or have a DEFAULT VALUE constraint (I can't show you how to do that unless you show me your table);
//This first method is psuedoCode to explain how to create your datatable. You need to do it in the way that makes sense for you.
public DataTable createDataTable(){
List<string> excludedColumns = new List<string>();
excludedColumns.Add("FieldToExclude");
//...
DataTable dt = new DataTable();
foreach(string col in getColumns(myTable)){
if(!excludedColumns.Contains(name)){
DataColumn dC = new DataColumn(name,type);
DataTable.Add(dC);
}
return dt;
}
public List<string> getColumns(string tableName)
{
List<string> ret = new List<string>();
using (SqlConnection conn = getConn())
{
conn.Open();
using (SqlCommand com = conn.CreateCommand())
{
com.CommandText = "select column_Name from information_schema.COLUMNS where table_name = #tab";
com.Parameters.AddWithValue("#tab", tableName);
SqlDataReader read = com.ExecuteReader();
While(read.Read()){
ret.Add(Convert.ToString(read[0]);
}
conn.Close();
}
return ret;
}
//Now, you have a DataTable that has all the columns you want to insert. Map them yourself in code by adding to the appropriate column in your datatable.
public bool isCopyInProgess = false;//not necessary - just part of my code
public void saveDataTable(string tableName, DataTable table)
{
using (SqlConnection conn = getConn())
{
conn.Open();
using (var bulkCopy = new SqlBulkCopy(conn))//, SqlBulkCopyOptions.KeepIdentity))//un-comment if you want to use your own identity column
{
// my DataTable column names match my SQL Column names, so I simply made this loop. However if your column names don't match, just pass in which datatable name matches the SQL column name in Column Mappings
foreach (DataColumn col in table.Columns)
{
//Console.WriteLine("mapping " + col.ColumnName+" ("+does_Column_Exist(col.ColumnName,"Item").ToString()+")");
bulkCopy.ColumnMappings.Add(col.ColumnName, "["+col.ColumnName+"]");
// Console.WriteLine("ok\n");
}
bulkCopy.BulkCopyTimeout = 8000;
bulkCopy.DestinationTableName = tableName;
bulkCopy.BatchSize = 10000;
bulkCopy.EnableStreaming = true;
//bulkCopy.SqlRowsCopied += BulkCopy_SqlRowsCopied;
//bulkCopy.NotifyAfter = 10000;
isCopyInProgess = true;
bulkCopy.WriteToServer(table);
}
conn.Close();
}
}
Also, use this as your bolumn checker:
public bool does_Column_Exist(string colName,string tableName)
{
bool ret = false;
using (SqlConnection conn = getConn())
{
conn.Open();
using (SqlCommand com = conn.CreateCommand())
{
com.CommandText = "select count(*) from information_schema.COLUMNS where column_name = #col and table_name = #tab";
com.Parameters.AddWithValue("#tab", tableName);
com.Parameters.AddWithValue("#col", colName);
ret = Convert.ToInt32(com.ExecuteScalar()) == 0 ? false : true;
}
conn.Close();
}
return ret;
}
Is there a specific reason you need C# for this? It seems like the path of least resistance would be to use SQL to do the job.
INSERT INTO table2
(column_name(s))
SELECT column_name(s)
FROM table1;
Ok, I have been having a problem the last few days with my database not updating. I can read the data fine and I'm not getting any exceptions either. I'm trying to update the database then I try to read values again after the update (during same run), and they still hold the original values, so it doesn't seem to be an issue with the database being copied to another folder (I'm using Copy if newer yet neither database is being updated).
Here is the code I'm using. As you can see I tried a few different approaches, none of which worked yet.
public void UpdateDatabaseInStock(string itemName, string tableName)
{
DataSet data = new DataSet("Items");
int val;
//get the file path to the database as a string
string dbfile =
new System.IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly().Location).DirectoryName +
"\\Database\\GameData.sdf";
//connect to the database
using (SqlCeConnection cntn = new SqlCeConnection("datasource=" + dbfile))
{
//create an adapter to pull all data from the table
using (SqlCeDataAdapter adpt = new SqlCeDataAdapter
("SELECT * FROM " + tableName + " WHERE Name LIKE '%" + itemName + "%'", cntn))
{
//put the data into a DataSet
adpt.Fill(data);
cntn.Close();
}
//fill the data from the Items table into a DataTable to return.
DataTable itemTable = data.Tables[0];
DataRow a = itemTable.Rows[0];
val = (short)a.ItemArray[3] - 1;
dbfile = "";
data.Dispose();
itemTable.Dispose();
SqlCeCommand cmd = new SqlCeCommand();
cmd.Connection = cntn;
cntn.Open();
cmd.CommandType = CommandType.Text;
cmd.CommandText = "UPDATE " + tableName + " SET [In Stock] = #Value WHERE [Name] = '#ItemName'";
//cmd.Parameters.Add("#Value", SqlDbType.SmallInt);
//cmd.Parameters["#Value"].Value = val;
//cmd.Parameters.Add("#ItemName", SqlDbType.NChar, 75);
//cmd.Parameters["#ItemName"].Value = itemName;
cmd.Parameters.AddWithValue("#Value", val);
cmd.Parameters.AddWithValue("#ItemName", itemName);
cmd.ExecuteNonQuery();
//close the conenction
cntn.Close();
cmd.Dispose();
}
}
Any ideas to get it to actually update?
Just a hunch (can't corroborate this on msdn): could it be that using nchar(75) adds spaces to the parameter, thereby causing the WHERE clause to fail?
I'm attempting to fill a DataTable with results pulled from a MySQL database, however the DataTable, although it is initialised, doesn't populate. I wanted to use this DataTable to fill a ListView. Here's what I've got for the setting of the DataTable:
public DataTable SelectCharacters(string loginName)
{
this.Initialise();
string connection = "0.0.0.0";
string query = "SELECT * FROM characters WHERE _SteamName = '" + loginName + "'";
if (this.OpenConnection() == true)
{
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataAdapter returnVal = new MySqlDataAdapter(query,connection);
DataTable dt = new DataTable("CharacterInfo");
returnVal.Fill(dt);
this.CloseConnection();
return dt;
}
else
{
this.CloseConnection();
DataTable dt = new DataTable("CharacterInfo");
return dt;
}
}
And for the filling of the ListView, I've got:
private void button1_Click(object sender, EventArgs e)
{
string searchCriteria = textBox1.Text;
dt = characterDatabase.SelectCharacters(searchCriteria);
MessageBox.Show(dt.ToString());
listView1.View = View.Details;
ListViewItem iItem;
foreach (DataRow row in dt.Rows)
{
iItem = new ListViewItem();
for (int i = 0; i < row.ItemArray.Length; i++)
{
if (i == 0)
iItem.Text = row.ItemArray[i].ToString();
else
iItem.SubItems.Add(row.ItemArray[i].ToString());
}
listView1.Items.Add(iItem);
}
}
Is there something I'm missing? The MessageBox was included so I could see if it has populated, to no luck.
Thanks for any help you can give.
Check your connection string and instead of using
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataAdapter returnVal = new MySqlDataAdapter(query,connection);
DataTable dt = new DataTable("CharacterInfo");
returnVal.Fill(dt);
this.CloseConnection();
return dt;
you can use this one
MySqlCommand cmd = new MySqlCommand(query, connection);
DataTable dt = new DataTable();
dt.load(cmd.ExecuteReader());
return dt;
Well, I ... can't figure out what you have done here so I'll paste you my code with which I'm filling datagridview:
1) Connection should look something like this(if localhost is your server, else, IP adress of server machine):
string connection = #"server=localhost;uid=root;password=*******;database=*******;port=3306;charset=utf8";
2) Query is ok(it will return you something), but you shouldn't build SQL statements like that.. use parameters instead. See SQL injection.
3) Code:
void SelectAllFrom(string query, DataGridView dgv)
{
_dataTable.Clear();
try
{
_conn = new MySqlConnection(connection);
_conn.Open();
_cmd = new MySqlCommand
{
Connection = _conn,
CommandText = query
};
_cmd.ExecuteNonQuery();
_da = new MySqlDataAdapter(_cmd);
_da.Fill(_dataTable);
_cb = new MySqlCommandBuilder(_da);
dgv.DataSource = _dataTable;
dgv.DataMember = _dataTable.TableName;
dgv.AutoResizeColumns();
_conn.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
finally
{
if (_conn != null) _conn.Close();
}
}
So, every time I want to display some content of table in mysql database I call this method, pass query string and datagridview name to that method. and, that is it.
For your sake, compare this example with your and see what can you use from both of it. Maybe, listview is not the best thing for you, just saying ...
hope that this will help you a little bit.
Debug your application and see if your sql statement/ connection string is correct and returns some value, also verify if your application is not throwing any exception.
Your connection string is invalid.
Set it as follows:
connection = "Server=myServer;Database=myDataBase;Uid=myUser;Pwd=myPassword;";
Refer: mysql connection strings
Here the following why the codes would not work.
Connection
The connection of your mySQL is invalid
Query
I guess do you want to search in the table, try this query
string query = "SELECT * FROM characters WHERE _SteamName LIKE '" + loginName + "%'";
notice the LIKE and % this could help to list all the data. for more details String Comparison Functions