I am writing a C# application
It is an offline test application
It imports a large data file in the form of a .csv file
This file is chosen by the user using a form
I then want to store the information contained in this .csv file in the form of a local database such that I can perform sql queries
I am using Visual Studio 2012
I have never setup an sql database before and only have limited experience using sql on existing databases
My attempt so far is:
Solution explorer > Add new file > Local Database (.sdf file)
Database Explorer > Tables > Create Table
I have then added column names for all the fields setting one as my primary key
I have attempted to add a single dataset to my data table with no luck
string dbfile = new System.IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly().Location).DirectoryName + "\\MyDatabase.sdf";
SqlCeConnection sqlConnection = new SqlCeConnection("datasource=" + dbfile);
SqlCeDataAdapter sqlAdapter = new SqlCeDataAdapter("select * from MyTable", sqlConnection);
AMCCoreSignalsDBDataSet sqlData = new AMCCoreSignalsDBDataSet();
sqlAdapter.Fill(sqlData);
string strCSVDataLine = "1,2,3,four"
sqlData.Tables[0].Rows.Add(new object[] { strCSVDataLine });
sqlAdapter.Update(sqlData);
sqlConnection.Close();
This code fails to work
How can I use C# to populate my database with the .csv data?
Is my method incorrect/incomplete?
Is there a better way to do this?
The reason I would like to use sql is because there is a lot of data. I could create a class structure to contain the data however it would also mean creating many different filter functions. Which SQL already contains...
Problems were due to blank values occurring in the .csv file
This was my fix
public void Import(string csvfname)
{
string password;
string cacheDatabase;
string connectionString;
System.IO.StreamReader objFile;
string strCommand;
string lineHeader;
string line;
string[] arrLineData;
cacheDatabase = new System.IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly().Location).DirectoryName + "\\MyDatabase.sdf"; ;
password = "";
connectionString = string.Format("DataSource=\"{0}\"; Password='{1}'", this.cacheDatabase, this.password);
objFile = new System.IO.StreamReader(csvfname);
lineHeader = objFile.ReadLine();
while (!objFile.EndOfStream)
{
line = objFile.ReadLine();
arrLineData = line.Split(',');
try
{
sqlConnection = new SqlCeConnection(connectionString());
strCommand = "INSERT INTO MyTable VALUES ('" + arrLineData[0] + "', '" + arrLineData[1] + "', '" + arrLineData[2] + "')";
SqlCeCommand sqlCommand = new SqlCeCommand(strCommand, sqlConnection);
sqlCommand.ExecuteNonQuery();
sqlConnection.Close();
}
catch (Exception exc)
{
MessageBox.Show("Error in Import(): " + exc.Message);
}
}
}
Related
I have to update multiple records in SQL Server table from C#. Below are the steps I have to follow and below is the code.
The code is working but the process is taking much longer than expected.
I need a quick way to update 10000 records, not sure whether Bulk Copy would work for Update.
I have seen the other answers which has Bulk insert to temp and then update..But that update has a single statement and here I need to update the records in DB based on Excel data and for this I have to loop each excel record.So how can I achieve faster update.
1) Read the Excel Data and copied the data into a data table
string strDirectory = string. Empty;
strDirectory = System.IO.Directory.GetCurrentDirectory() + "\\" + "Filename.xlsx";
string Connection String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source = " + strDirectory + "; Extended Properties = \"Excel 12.0;HDR=YES;IMEX=1\"";
using (OleDbConnection conn = new OleDbConnection(Connection String))
{
conn.Open();
DataTable schemaTable = conn.GetOleDbSchemaTableOleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
DataRow schemaRow = schemaTable. Rows[0];
string sheet = schemaRow["TABLE_NAME"].ToString();
string query = "SELECT * FROM [" + sheet + "]";
OleDbDataAdapter daexcel = new OleDbDataAdapter(query, conn);
daexcel.Fill(dt);
conn.Close();
}
2) Doing some manipulations on datatable data before updating into table.
string strsqlst = string. Empty;
using (SqlConnection sqlConn = new SqlConnection(Connectionstring))
{
sqlConn.Open();
SqlCommand cmd;
StringBuilder sb = new StringBuilder();
sb.AppendLine("DataTable content:");
foreach (DataRow row in dt.Rows)
{
if (row.ItemArray[0].ToString() == "")
break;
strsqlst = "Update table Set col1= " + row.ItemArray[4].ToString() + " ,col2= " + row.ItemArray[5].ToString() + " where <Condition>'";
cmd = new SqlCommand(strsqlst, sqlConn);
cmd.CommandType = CommandType.Text;
cmd.ExecuteNonQuery();
}
sqlConn.Close();
}
The SqlCommand can be a whole SQL batch and is not limited to a single statement. So you can create a single large batch with 10,000 UPDATE statements, or divide it into for example 20 batches of 500 each.
In other words, you can create a single command with CommandText like this:
UPDATE [T] SET Col1='Value1', Col2='Value2' WHERE [Id] = 1;
...
UPDATE [T] SET Col1='Value999', Col2='Value1000' WHERE [Id] = 500;
That said, you should use parameters for all data values (to ensure SQL injection is not possible).
If you want to handle any errors (updates failing due to invalid data) you will need something a bit more sophisticated.
I have tried nearly every solution on this website, but I can't solve this problem. I have data that has been retrieved from a database through an ODBC connection. The data is there. It will go into a Data Grid View just fine, but I can't get this data to go into my local SQL database. Please tell me what I'm doing wrong.
public partial class frmNorth : Form
{
// variables for the connections
private OdbcConnection epnConnection = new OdbcConnection();
private SqlConnection tempDbConnection = new SqlConnection();
public frmNorth()
{
InitializeComponent();
// This is for the ePN DB
epnConnection.ConnectionString = #"Dsn=ePN; uid=username; pwd=myPa$$Word";
// This is for the local DB
tempDbConnection.ConnectionString = #"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\TempDB.mdf;Integrated Security=True";
}
private void btnLoadData_Click(object sender, EventArgs e)
{
try
{
//===This part works just fine===============================================================
epnConnection.Open();
string epnQuery = "SELECT FNCL_SPLIT_REC_ID, PROJ_ID, SALES_SRC_PRC " +
"FROM PROJ_FNCL_SPLIT " +
"WHERE PROJ_ID=" + textBox1.Text + "";
OdbcCommand epnCommand = new OdbcCommand(epnQuery, epnConnection);
epnCommand.CommandTimeout = 0;
//This connects the data to the data table
OdbcDataAdapter da = new OdbcDataAdapter(epnCommand);
DataTable dt = new DataTable();
da.Fill(dt);
dataGridView1.DataSource = dt;
//===========================================================================================
//======The part below is the part that wont work. The data wont go into the SQL database====
tempDbConnection.Open();
string tempSql = "";
for (int i = 0; i < dt.Rows.Count; i++)
{
tempSql = "INSERT INTO tblTemp (FNCL_SPLIT_REC_ID, PROJ_ID, SALES_SRC_PRC) VALUES ('"
+ dt.Rows[i]["FNCL_SPLIT_REC_ID"].ToString().Trim() + "','"
+ dt.Rows[i]["PROJ_ID"].ToString().Trim() + "','"
+ dt.Rows[i]["SALES_SRC_PRC"].ToString().Trim() + "');";
SqlCommand tempCommand = new SqlCommand(tempSql, tempDbConnection);
tempCommand.ExecuteNonQuery();
}
// There are no errors. The data just doesn't save to the database.
//===========================================================================================
epnConnection.Close();
tempDbConnection.Close();
}
catch (Exception ex)
{
epnConnection.Close();
tempDbConnection.Close();
MessageBox.Show("Error " + ex);
}
}
}
}
//+++++++++++++++++++This is what the table looks like+++++++++++++++++++++++++++++++++++++++++++++++
CREATE TABLE [dbo].[tblTemp] (
[FNCL_SPLIT_REC_ID] INT NOT NULL,
[PROJ_ID] NCHAR (10) NULL,
[SALES_SRC_PRC] MONEY NULL,
PRIMARY KEY CLUSTERED ([FNCL_SPLIT_REC_ID] ASC)
Like I said no errors come up. The data just doesn't save to the database.
"INSERT INTO tblTemp (FNCL_SPLIT_REC_ID, PROJ_ID, SALES_SRC_PRC) VALUES ("
+ dt.Rows[i]["FNCL_SPLIT_REC_ID"].ToString().Trim() + ",'"
+ dt.Rows[i]["PROJ_ID"].ToString().Trim() + "',"
+ dt.Rows[i]["SALES_SRC_PRC"].ToString().Trim() + ");";
Removed the ' ' between FNCL_SPLIT_REC_ID as it is int and SALES_SRC_PRC since it is money.
I found no error on code you implemented. I found that connection definition for mdf file was wrong.
|DataDirectory| setting path to the folder where application run. In this case if we run in Debug mode, it will create separate application exe in Debug\bin folder with application resources like .mdf files. or in Release mode it will create particular folders inside release folder. So you need to change Database File name for database connection or you need to give entire directory path for connection string. Example
tempDbConnection.ConnectionString = #"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\TempDB.mdf;Integrated Security=True";
}
replace
tempDbConnection.ConnectionString = #"Data Source=(LocalDB)\v11.0;AttachDbFilename=C:\Users\Promod\Documents\Visual Studio 2012\Projects\Contribution1\Contribution1\bin\Debug\TempDB.mdf;Integrated Security=True";
Ok, I have been having a problem the last few days with my database not updating. I can read the data fine and I'm not getting any exceptions either. I'm trying to update the database then I try to read values again after the update (during same run), and they still hold the original values, so it doesn't seem to be an issue with the database being copied to another folder (I'm using Copy if newer yet neither database is being updated).
Here is the code I'm using. As you can see I tried a few different approaches, none of which worked yet.
public void UpdateDatabaseInStock(string itemName, string tableName)
{
DataSet data = new DataSet("Items");
int val;
//get the file path to the database as a string
string dbfile =
new System.IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly().Location).DirectoryName +
"\\Database\\GameData.sdf";
//connect to the database
using (SqlCeConnection cntn = new SqlCeConnection("datasource=" + dbfile))
{
//create an adapter to pull all data from the table
using (SqlCeDataAdapter adpt = new SqlCeDataAdapter
("SELECT * FROM " + tableName + " WHERE Name LIKE '%" + itemName + "%'", cntn))
{
//put the data into a DataSet
adpt.Fill(data);
cntn.Close();
}
//fill the data from the Items table into a DataTable to return.
DataTable itemTable = data.Tables[0];
DataRow a = itemTable.Rows[0];
val = (short)a.ItemArray[3] - 1;
dbfile = "";
data.Dispose();
itemTable.Dispose();
SqlCeCommand cmd = new SqlCeCommand();
cmd.Connection = cntn;
cntn.Open();
cmd.CommandType = CommandType.Text;
cmd.CommandText = "UPDATE " + tableName + " SET [In Stock] = #Value WHERE [Name] = '#ItemName'";
//cmd.Parameters.Add("#Value", SqlDbType.SmallInt);
//cmd.Parameters["#Value"].Value = val;
//cmd.Parameters.Add("#ItemName", SqlDbType.NChar, 75);
//cmd.Parameters["#ItemName"].Value = itemName;
cmd.Parameters.AddWithValue("#Value", val);
cmd.Parameters.AddWithValue("#ItemName", itemName);
cmd.ExecuteNonQuery();
//close the conenction
cntn.Close();
cmd.Dispose();
}
}
Any ideas to get it to actually update?
Just a hunch (can't corroborate this on msdn): could it be that using nchar(75) adds spaces to the parameter, thereby causing the WHERE clause to fail?
I'm trying to translate some perl code into C# and I'm having some trouble with the following.
After establishing a sql server connection and executing the select statement, how do I reference the different elements in the table columns. For example, in Perl it looks like:
my $dbh = DBI -> connect( NAME, USR, PWD )
or die "Failed to connect to database: " . DBI->message;
my $dbname = DB_NAME;
my $dbschema = DB_SCHEMA;
my $sql = qq{select a,b,c,d,e,f,g,h,i,...
from $dbname.$dbschema.package p
join $dbname.$dbschema.package_download pd on p.package_id = pd.package_id
join $dbname.$dbschema.download d on pd.download_id = d.download_id
where p.package_name = '$package'
--and ds.server_address like 'tcp/ip'
order by a,b,c,d,..};
my $sth = $dbh -> prepare( $sql )
or die "Failed to prepare statement: " . $dbh->message;
$sth -> execute()
or die "Failed to execute statement: " . $sth->message;
#now to go through each row in result table
while ( #data = $sth->fetchrow_array() )
{
print "$data[0]";
# If source server FTP is not already open, make new FTP
if ( $data[0] != $src_id )
{
if ( $src_ftp )
{ $src_ftp -> quit; }
$src_ftp = make_ftp( $data[1], $data[2], $data[3], $data[18], $data[19], $data[20] );
$src_id = $data[0];
}
}
so far I've got it down to
string db = NAME;
string myConnectionString = "Data Source=ServerName;" + "Initial Catalog=" + db + "User id=" + ODBC_USR + "Password=" + PWD
SqlConnection myConnection = new SqlConnection(myConnectionString);
string myInsertQuery = "select a,b,c,d,e,f,g,h,i,...
from $dbname.$dbschema.package p
join $dbname.$dbschema.package_download pd on p.package_id = pd.package_id
join $dbname.$dbschema.download d on pd.download_id = d.download_id
where p.package_name = '$package'
--and ds.server_address like 'tcp/ip'
order by a,b,c,d,..";
SqlCommand myCommand = new SqlCommand(myInsertQuery);
myCommand.Connection = myConnection;
myConnection.Open();
myCommand.ExecuteNonQuery();
myCommand.Connection.Close();
but how do I reference the columns like data[0] and data[1] in C#. Sorry I'm new to both languages so my background is severely lacking. Thanks!
You could reference your column directly by its column name or by numeric order (it starts with 0 as the first column) either through a DataTable, DataSet, DataReader or a specific DataRow.
For the sake of example i'll use a DataTable here and I will name it as dt and let's say we want to reference the first row then you could reference it with the following Syntax/Format:
dt[RowNumber]["ColumnName or Column Number"].ToString();
For example:
dt[0]["a"].ToString();
Or by number the first column with be 0 like:
dt[0][0].ToString();
And use Parameters by the way because without which it would be susceptible to SQL Injection. Here's a more complete code below:
string db = NAME;
string myConnectionString = "Data Source=ServerName;" + "Initial Catalog=" + db + "User id=" + ODBC_USR + "Password=" + PWD
using (SqlConnection connection = new SqlConnection(myConnectionString))
{
string mySelectQuery = #"SELECT a,b,c,d,e,f,g,h,i,...
FROM package p
JOIN package_download pd on p.package_id = pd.package_id
join download d on pd.download_id = d.download_id
WHERE p.package_name = #PackageName
AND ds.server_address LIKE 'tcp/ip%'
ORDER by a,b,c,d";
try
{
connection.Open();
using (SqlDataAdapter da = new SqlDataAdapter(mySelectQuery, connection))
{
using (SqlCommand cmd = new SqlCommand())
{
da.SelectCommand.Parameters.AddWithValue("#PackageName", txtPackage.Text);
DataTable dt = new DataTable();
da.Fill(dt);
if (dt.Rows.Count>0) // Make sure there is something in your DataTable
{
String aVal = dt[0]["a"].ToString();
String bVal = dt[0]["b"].ToString();
// You'll be the one to fill up
}
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
I change your LIKE 'tcp/ip' to LIKE 'tcp/ip%' by the way which is the more appropriate one of using LIKE.
you can use ado.net entity data table to reference the tables in your sql server. I don't know if you're asking exactly this but it may help. because direct referencing to sql server is not possible as far as i know.
I have a C# webservice on a Windows Server that I am interfacing with on a linux server with PHP. The PHP grabs information from the database and then the page offers a "more information" button which then calls the webservice and passes in the name field of the record as a parameter. So i am using a WHERE statement in my query so I only pull the extra fields for that record. I am getting the error:
System.Data.SqlClient.SqlException:Invalid column name '42'
Where 42 is the value from the name field from the database.
my query is
string selectStr = "SELECT name, castNotes, triviaNotes FROM tableName WHERE name =\"" + show + "\"";
I do not know if it is a problem with my query or something is wrong with the database, but here is the rest of my code for reference.
NOTE: this all works perfectly when I grab all of the records, but I only want to grab the record that I ask my webservice for.
public class ktvService : System.Web.Services.WebService {
[WebMethod]
public string moreInfo(string show) {
string connectionStr = "MyConnectionString";
string selectStr = "SELECT name, castNotes, triviaNotes FROM tableName WHERE name =\"" + show + "\"";
SqlConnection conn = new SqlConnection(connectionStr);
SqlDataAdapter da = new SqlDataAdapter(selectStr, conn);
DataSet ds = new DataSet();
da.Fill(ds, "tableName");
DataTable dt = ds.Tables["tableName"];
DataRow theShow = dt.Rows[0];
string response = "Name: " + theShow["name"].ToString() + "Cast: " + theShow["castNotes"].ToString() + " Trivia: " + theShow["triviaNotes"].ToString();
return response;
}
}
Quick solution:
I believe you need single quotes in your selectStr:
string selectStr =
"SELECT name, castNotes, triviaNotes FROM tableName WHERE name = '" + show + "'";
More information:
In .NET, you'll want to be sure you close out any connections explicitly when you no longer need them. The easiest way to do this is to wrap using statements around any types that implement IDisposable, such as SqlConnection in this case:
using(SqlConnection conn = new SqlConnection(connectionStr))
{
SqlDataAdapter da = new SqlDataAdapter(selectStr, conn);
DataSet ds = new DataSet();
da.Fill(ds, "tableName");
DataTable dt = ds.Tables["tableName"];
DataRow theShow = dt.Rows[0];
string response = "Name: " + theShow["name"].ToString() + "Cast: " + theShow["castNotes"].ToString() + " Trivia: " + theShow["triviaNotes"].ToString();
return response;
}
Additionally, it looks like your code could be easily subject to SQL injection. What if someone submits a form with the value: fake name' OR 1=1;DROP DATABASE someDbName;--?
You'll want to take advantage of SQL parameters, something like:
SqlCommand cmd = new SqlCommand(
"SELECT name, castNotes, triviaNotes FROM tableName WHERE name = #show", conn);
cmd.Parameters.AddWithValue("#show", show);
Shouldn't the WHERE clause be WHERE name = '" + show + "'"; Strings should be enclosed in single quotes and not double quotes for SQL statements.
Also the System.Data.SqlClient namespace is for SQL Server and not MySQL. See MySQL official docs for connecting to MySQL via C#.