[BEGINNER]
Sorry for my English.
I've been working for a new company for a short time and I consider myself a novice in BDD, since apart from school I haven't practiced since.
[ENVIRONMENT]
I am developing in C# an existing software that I need to optimize. With a big constraint, it is not to modify the bases since many customers already use them and will not want to change their server. In general if he uses ORACLE it is because he also uses it for other software.
The software used System.Data.OracleClient, but the latter is obsolete, I started implementing Oracle.ManagedDataAccess.Client. I can communicate, read and write data with the database, but I can't send a large amount of data quickly.
On site we have a server with a remote base which is therefore with Oracle, or MySQL (what interests me for the moment is Oracle)
And there are local workstations that use an SQLite database and must be able to operate without the remote database.
By default everything is saved locally and I have synchronizations to do if the remote database is accessible. It is therefore possible to have a large amount of data to transfer. (there are measurement records)
[FOR MY TESTS]
I recovered one of the customer databases in order to do tests with real data and one of the tables with more than 650,000 rows.
Transferring this line by line takes a lot of time, even if I don't make disconnection connections on each line of course.
I wanted to try to send parameter blocks, but I can't do it at the moment.
[MY RESEARCH]
I keep looking, but what I found was either using paid DLLs or I didn't understand their use.
[MY CODE]
For the moment I left on it, really in test:
public void testOracle()
{
try
{
if (Connection == null)
{
Connection = OracleConnection();
}
string commandString = "INSERT INTO GRAPHE (ID, MESUREE_ID, DATE_MESURE, POINT_GRAPHE, D0, D1) VALUES (:IDp, :MESUREE_IDp, to_timestamp( :DATE_MESUREp ,'DD/MM/RR HH24:MI:SSXFF'), :POINT_GRAPHEp, :D0p, :D1p)";
int _ID = 1;
int _MESUREE_ID = 9624;
string _DATE_MESURE = "16/12/ 08 00:00:00,000000000";
int _POINT_GRAPHE = 1229421394;
int[] _D0 = 0;
int[] _D1 = 0;
using (OracleCommand command = new OracleCommand(commandString, Connection))
{
using (var transaction = Connection.BeginTransaction())
{
for (int i = 0; i < _ID.Length; i++)
{
command.Parameters.Add("IDp", OracleDbType.Decimal).Value = _ID;
command.Parameters.Add("MESUREE_IDp", OracleDbType.Decimal).Value = _MESUREE_ID];
command.Parameters.Add("DATE_MESUREp", OracleDbType.Varchar2).Value = _DATE_MESURE[i];
command.Parameters.Add("POINT_GRAPHEp", OracleDbType.Decimal).Value = _POINT_GRAPHE[i];
command.Parameters.Add("DOS10p", OracleDbType.Decimal).Value = _D0[i];
command.Parameters.Add("DOS07p", OracleDbType.Decimal).Value = _D1[i];
command.ExecuteNonQuery();
}
transaction.Commit();
}
}
}
catch (Exception ex)
{
}
Connection.Close();
}
public OracleConnection OracleConnection()
{
string serveur_name = "192.168.0.1";
string database_name = "oracle.dev";
string user_name = "name";
string password = "pass";
string oraclePort = "1521";
OracleConnection _con = null;
try
{
string connectionString = "Data Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=" + serveur_name + ")(PORT=" + oraclePort + ")) (CONNECT_DATA=(SERVICE_NAME=" + database_name + "))); User Id=" + user_name + ";Password=" + password + ";";
_con = new OracleConnection(connectionString);
try
{
_con.Open();
}
catch (Exception e)
{
}
}
catch (Exception e)
{
}
return _con;
}
}```
then you can create a DataTable and do bulk insert
using (var bulkCopy = new OracleBulkCopy(yourConnectionString, OracleBulkCopyOptions.UseInternalTransaction))
{
bulkCopy.DestinationTableName = "GRAPHE";
bulkCopy.WriteToServer(dataTable);
}
You can also use the BatchSize property as per your requirement.
Related
Is the code below implementing the secure way to retrieve the data from database?
help me please, I don't understand about SQL Injection. Someone told me this code can easily get injected. If yes, can somebody explain it? Thank you.
public int CheckID(string column, string table, string wheres)
{
int i = 0;
sqlcon = ConnectToMain();
string sqlquery = "SELECT "+column+" FROM "+table+" "+wheres+"";
using (sqlcon)
{
sqlcon.Open();
SqlCommand sqlcom = new SqlCommand(sqlquery, sqlcon);
using (sqlcom)
{
SqlDataReader dr = sqlcom.ExecuteReader();
dr.Read();
if (dr.HasRows)
{
i = dr.GetInt32(0);
}
else
{
i = 0;
}
}
sqlcon.Close();
}
return i;
}
This code has far too many problems.
Table, column and criteria are passed as strings and concatenated, which means that the code is prone to SQL injection.
Database details like table, column criteria are spilled into the function's caller. Are you going to use this method to query anything other than a Visitor table?
A reader is used when only a single value is wanted.
The connection is created outside the using block and stored in a field.
This is definitelly a memory leak and probably a connection leak as well. Just create the connection locally.
A simple command call fixes all of these problems:
public int CheckIDVisitor(visitorName)
{
string query = "SELECT ID FROM Visitors where Name=#name";
using (var sqlConn=new SqlConnection(Properties.Default.MyDbConnectionString))
using( var cmd=new SqlCommand(query,sqlConn))
{
var cmdParam=cmd.Parameters.Add("#name",SqlDbType.NVarChar,20);
cmdParam.Value=visitorName;
sqlConn.Open();
var result=(int?)cmd.ExecuteScalar();
return result??0;
}
}
You could also create the command in advance and store it in a field. You can attach the connection to the command each time you want to execute it:
public void InitVisitorCommand()
{
string query = "SELECT ID FROM Visitors where Name=#name";
var cmd=new SqlCommand(query,sqlConn);
var cmdParam=cmd.Parameters.Add("#name",SqlDbType.NVarChar,20);
_myVisitorCommand=cmd;
}
...
public int CheckIDVisitor(visitorName)
{
using (var sqlConn=new SqlConnection(Properties.Default.MyDbConnectionString))
{
_myVisitorCommand.Parameters.["#name"]Value=visitorName;
_myVisitorCommand.Connection=sqlConn;
sqlConn.Open();
var result=(int?)cmd.ExecuteScalar();
return result??0;
}
}
An even better option would be to use a micro-ORM like Dapper.Net to get rid of all this code:
public int CheckIDVisitor(visitorName)
{
using (var sqlConn=new SqlConnection(Properties.Default.MyDbConnectionString))
{
string sql = "SELECT ID FROM Visitors WHERE name=#name"
var result = conn.Query<int?>(sql, new { name = visitorName);
return result??0;
}
}
Or
public int[] CheckIDVisitors(string []visitors)
{
using (var sqlConn=new SqlConnection(Properties.Default.MyDbConnectionString))
{
string sql = "SELECT ID FROM Visitors WHERE name IN #names"
var results = conn.Query<int?>(sql, new { names = visitors);
return results.ToArray();
}
}
As the server requires a connection for each database, the answers I've found in SO don't work at all.
For some tables I have to make some calculations before copying the rows, but some I can copy whole table. And finally it's an automation in my program that I've wrote.
Oldcn is the connection to old database, Newcn for the new database.
For the tables that I can copy whole I wrote this procedure below.
Is there a better an short way to do this job? (It works on background)
private string[] CopyTva(MySqlConnection Oldcn, MySqlConnection Newcn, string[] res, DoWorkEventArgs we,string msg)
{
int counter = int.Parse(res[1]);
MySqlCommand cmd = new MySqlCommand("SELECT * FROM tvaval", Oldcn);
MySqlCommand cmd1 = new MySqlCommand("INSERT INTO tvaval (id,tvavalue,cr_user,cr_date,up_user,up_date) VALUES (#id,#tvavalue,#cr_user,#cr_date,#up_user,#up_date)", Newcn);
MySqlDataReader rd = null;
try
{
rd = cmd.ExecuteReader();
while (rd.Read())
{
cmd1.Parameters.AddWithValue("#id", rd["id"].ToString());
//bla bla same as above
try
{
cmd1.ExecuteNonQuery();
}
catch (MySqlException e)
{
rd.Dispose();
res[2] = "Erreur:TVA " + e.Message.ToString();
return res;
}
++counter;
bgw.ReportProgress((counter * 100) / DbTotalRow,msg);
cmd1.Parameters.Clear();
}
rd.Dispose();
}
catch (MySqlException e)
{
res[2] = "Erreur:TVA " + e.Message.ToString();
return res;
}
res[0] = "1";res[1] = counter.ToString();res[2] = "";
return res;
}
MySQL doesn't require a separate connection for each database. A connection has a default database, but you're allowed to specify the database name explicitly before the table name, and this will override the default. So you can do this with a single query:
INSERT INTO newdb.tvaval
SELECT * FROM olddb.tvaval
My MySQL connection throws null reference although this code worked well one year ago.
The line where the debugger indicates the exception contains
"connection = new MySqlConnection(connectionString);":
DBConnect MySqlConnection = new DBConnect();
string[] resultArray = MySqlConnection.GetValidUser(
"tbl_user",
tbEmail.Text,
tbPassword.Text
);
//THROWS null reference exception in method 'private bool OpenConnection()'
//A first chance exception of type 'System.ArgumentException' occurred in System.Data.dll
This is my DBConnect class:
class DBConnect
{
private MySqlConnection connection;
private string server;
private string database;
private string uid;
private string password;
public DBConnect()
{
server = "x.x.x.x";
database = "...";
uid = "...";
password = "...";
string connectionString = "SERVER=" + server + ";" +
"DATABASE=" + database + ";" +
"UID=" + uid + ";" +
"PASSWORD=" + password + ";";
connection = new MySqlConnection(connectionString);
}
private bool OpenConnection()
{
try
{
connection.Open();
return true;
}
catch (MySqlException ex)
{
switch (ex.Number)
{
case 0:
MessageBox.Show("Cannot connect to MySQL server.");
break;
case 1045:
MessageBox.Show("Invalid username or password.");
break;
}
return false;
}
}
public string[] GetValidUser(string dbTable, string dbUsername, string dbPassword)
{
string query = "SELECT id,email,password FROM " + dbTable +
" WHERE email='" + dbUsername +
"' AND password='" + dbPassword + "'";
string[] resultArray = new string[3];
if (this.OpenConnection() == true)
{
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataReader dataReader = cmd.ExecuteReader();
while (dataReader.Read())
{
resultArray[0] = dataReader.GetInt32(0).ToString();
resultArray[1] = dataReader.GetString(1);
resultArray[2] = dataReader.GetString(2);
}
dataReader.Close();
this.CloseConnection();
}
return resultArray;
}
}
The original code for the database class can be found here.
This is not an answer to the NullReferenceException - we're still working through that in the comments; this is feedback for the security parts.
The first thing we can look at is SQL injection; this is very easy to fix - see below (note I've tidied some other things too)
// note: return could be "bool" or some kind of strongly-typed User object
// but I'm not going to change that here
public string[] GetValidUser(string dbUsername, string dbPassword)
{
// no need for the table to be a parameter; the other two should
// be treated as SQL parameters
string query = #"
SELECT id,email,password FROM tbl_user
WHERE email=#email AND password=#password";
string[] resultArray = new string[3];
// note: it isn't clear what you expect to happen if the connection
// doesn't open...
if (this.OpenConnection())
{
try // try+finally ensures that we always close what we open
{
using(MySqlCommand cmd = new MySqlCommand(query, connection))
{
cmd.Parameters.AddWithValue("email", dbUserName);
// I'll talk about this one later...
cmd.Parameters.AddWithValue("password", dbPassword);
using(MySqlDataReader dataReader = cmd.ExecuteReader())
{
if (dataReader.Read()) // no need for "while"
// since only 1 row expected
{
// it would be nice to replace this with some kind of User
// object with named properties to return, but...
resultArray[0] = dataReader.GetInt32(0).ToString();
resultArray[1] = dataReader.GetString(1);
resultArray[2] = dataReader.GetString(2);
if(dataReader.Read())
{ // that smells of trouble!
throw new InvalidOperationException(
"Unexpected duplicate user record!");
}
}
}
}
}
finally
{
this.CloseConnection();
}
}
return resultArray;
}
Now, you might be thinking "that's too much code" - sure; and tools exist to help with that! For example, suppose we did:
public class User {
public int Id {get;set;}
public string Email {get;set;}
public string Password {get;set;} // I'll talk about this later
}
We can then use dapper and LINQ to do all the heavy lifting for us:
public User GetValidUser(string email, string password) {
return connection.Query<User>(#"
SELECT id,email,password FROM tbl_user
WHERE email=#email AND password=#password",
new {email, password} // the parameters - names are implicit
).SingleOrDefault();
}
This does everything you have (including safely opening and closing the connection), but it does it cleanly and safely. If it method returns a null value for the User, it means no match was found. If a non-null User instance is returned - it should contain all the expected values just using name-based conventions (meaning: the property names and column names match).
You might notice that the only code that remains is actually useful code - it isn't boring plumbing. Tools like dapper are your friend; use them.
Finally; passwords. You should never store passwords. Ever. Not even once. Not even encrypted. Never. You should only store hashes of passwords. This means that you can never retrieve them. Instead, you should hash what the user supplies and compare it to the pre-existing hashed value; if the hashes match: that's a pass. This is a complicated area and will require significant changes, but you should do this. This is important. What you have at the moment is insecure.
Among other things, it sounds like you have problems with the connection string - from comments:
While "connection = new MySqlConnection(); connection.ConnectionString = connectionString;" throws an exception the statement "connection = new MySqlConnection();" does not...
The difference here is simply: in the latter you aren't setting the connection string - so it sounds like your connection string is not correctly escaping the values (most likely, the password); you could try:
var cs = new DbConnectionStringBuilder();
cs["SERVER"] = server;
cs["DATABASE"] = database;
cs["UID"] = uid;
cs["PASSWORD"] = password;
var connectionString = cs.ConnectionString;
Okay, I've built a SQL Server database which is being accessed and manipulated by an ASP.NET UI (which I have also developed) to allow others at work to search the DB easily. The database holds data on numerous locations where we have network equipment operating.
I have been asked to build into the UI the capability to query the database for multiple IP addresses - e.g. a user will enter into a textbox "192.168.1.0, 18.15.156.4", click enter and be presented with the results in the gridview. Multiple IP addresses will be separated with a ,.
The code below basically strips out space characters, looks for a , (to determine how many ips there are to query) and if found puts them into an array. A for loop then puts each array item into its own session variable, and from there they are ready to be queried:
protected void siteSearchButton_Click(object sender, EventArgs e)
{
//checks IP search textbox is empty
if (ipQueryTextBox.Text != null)
{
searchErrorLabel.Visible = false;
string addresses = ipQueryTextBox.Text;
//checks for any blank spaces in the addresses variable
if (addresses.Contains(" "))
{
addresses = addresses.Replace(" ", "");
}
//sceens for multiple search items by looking for a ','
if (addresses.Contains(","))
{
//declaring int variables to be used in each of the respective loops
int j = 0;
string[] IParray = addresses.Split(',');
//if i is equal to the length of the "addresses" variable, execute the for loop enclosed
foreach (string s in IParray)
{
Session["IP" + j] = IParray[j];
j = j + 1;
}
}
}
}
As the number of ips to be queried against the database is dynamic I have come to the conclusion that I will have to use C# code (which I'm okay with), but as far as what I've got so far below, I'm unsure how to go about querying the db 'x' amount of times using code presumably I'll need to use a while loop, is anyone able to offer some insight?
//****THE SQL COMMAND BELOW NEEDS ADAPTING TO ALLOW MULTIPLE QUERIES FOR EACH OF THE VALUES STORED IN IParray ---> each session variable
if()
{
//opens a new sqlconnection to read and populate edit textboxes from the Inventory database
using (SqlConnection connection = new SqlConnection("Data Source=localhost;Initial Catalog=Inventory;Integrated Security=True"))
{
//declares SQLCommand type named 'command' and assigns it a string value of SQL code
SqlCommand command =
new SqlCommand("select * from LOCATION WHERE IP_ADDRESS=#IP_ADDRESS", connection);
//outlines parameters
command.Parameters.Add("#IP_ADDRESS", System.Data.SqlDbType.VarChar);
command.Parameters["#IP_ADDRESS"].Value = Session["IP"+j];;
connection.Open();
//opens database connection
SqlDataReader read = command.ExecuteReader();
//while loop will convert each record to string value and print entry into textbox. Will continue untill it runs out of lines
while (read.Read())
{
}
read.Close();
}
}
Instead of using multiple queries, just use SQL's IN clause. It does require a little bit more work to set the query parameters though.
string[] ips = new string[] { "192.168.0.1", "192.168.0.2", "192.168.0.3" };
string[] parameters = ips.Select(
(ip, index) => "#ip" + index.ToString()
).ToArray();
string commandFormat = "SELECT * FROM LOCATION WHERE IP_ADDRESS IN ({0})";
string parameterText = string.Join(",", parameters);
string commandText = string.Format(commandFormat, parameterText);
using (SqlCommand command = new SqlCommand(commandText)) {
for(int i = 0; i < parameters.Length; i++) {
command.Parameters.AddWithValue(parameters[i], ips[i]);
}
}
In the above example, the generated command will be SELECT * FROM LOCATION WHERE IP_ADDRESS IN (#ip1,#ip2,#ip3), and the parameter values will be set accordingly.
(The above solution was very much inspired by this answer.)
First thing - why create multiple session objects when you needs just one to store the values?
I'd try to change your code like this:
if (ipQueryTextBox.Text != null)
{
searchErrorLabel.Visible = false;
string addresses = ipQueryTextBox.Text;
addresses = addresses.Replace(" ", "");
addresses = addresses.Replace(",", "','");
Session["addresses"] = addresses;
}
For the SQL part, you can now easily utilize SQL IN operator, documented here for instance: http://www.w3schools.com/sql/sql_in.asp
SqlCommand command = new SqlCommand("select * from LOCATION WHERE IP_ADDRESS IN (#IP_ADDRESSES)", connection);
command.Parameters.AddWithValue("IP_ADDRESSES", Session["addresses"]);
This should probably work, but I haven't tested it, so it may require some tweaking. Hope you get the idea.
why do you need parameters at all.
//get IP address from UI;
string IPAddress = ipQueryTextBox.Text; //e.g. "192.168.0.1,192.168.0.2,192.168.0.3"
string commandFormat = "SELECT * FROM LOCATION WHERE IP_ADDRESS IN ('" + string.Join("','", IPAddress.split(",")) + "')";
now execute the query
Thank you to everyone who has responded, here is the solution I came up with derived from the answers above:
protected void siteSearchButton_Click(object sender, EventArgs e)
{
//checks IP search textbox is empty
if (ipQueryTextBox.Text != null)
{
searchErrorLabel.Visible = false;
string addresses = ipQueryTextBox.Text;
//checks for any blank spaces in the addresses variable
if (addresses.Contains(" "))
{
addresses = addresses.Replace(" ", "");
}
//sceens for multiple search items by looking for a ','
if (addresses.Contains(","))
{
string[] IParray = addresses.Split(',');
string[] Parameters= IParray.Select((IP, index)=>"#ip"+ index.ToString()).ToArray();
string commandformat ="SELECT * FROM LOCATION WHERE IP_ADDRESS IN ({0})";
string parametertxt= string.Join(",",Parameters);
string commandtxt= string.Format(commandformat,parametertxt);
//creates an SQL connection "connection" opens the connection creates the sql command to be executed & binds and refreshes the gridview
using (SqlConnection connection = new SqlConnection("Data Source=localhost;Initial Catalog=Inventory;Integrated Security=True"))
{
SqlDataReader reader = null;
connection.Open();
SqlCommand command = new SqlCommand(commandtxt, connection);
for(int i =0; i<Parameters.Length; i++)
{
command.Parameters.AddWithValue(Parameters[i],IParray[i]);
}
reader = command.ExecuteReader();
browseSiteGridView.DataSource = reader;
browseSiteGridView.DataBind();
reader.Close();
connection.Close();
}
}
else
{
//creates an SQL connection "connection" opens the connection creates the sql command to be executed & binds and refreshes the gridview
string commandtxt="SELECT * FROM LOCATION WHERE IP_ADDRESS ='"+addresses+"'";
using (SqlConnection connection = new SqlConnection("Data Source=localhost;Initial Catalog=Inventory;Integrated Security=True"))
{
SqlDataReader reader = null;
connection.Open();
SqlCommand command = new SqlCommand(commandtxt, connection);
reader = command.ExecuteReader();
browseSiteGridView.DataSource = reader;
browseSiteGridView.DataBind();
reader.Close();
connection.Close();
}
}
}
I have problem with an INSERT statement. I get data from gridviev and it works fine, but I can't insert them into table
private void button3_Click(object sender, EventArgs e)
{
int amorplanid = 0;
int idn = 0;
DateTime datum;
double interest = 0;
double principal = 0;
double payment = 0;
double newprincipal = 0;
string nizz = "";
string[] niz= new string[7];
for (int x = 0; x < dataGridView1.Rows.Count-1; x++)
{
for (int j = 0; j < dataGridView1.Rows[x].Cells.Count; j++)
{
nizz += dataGridView1.Rows[x].Cells[j].Value.ToString()+".";
}
niz = nizz.Split('.');
amorplanid = System.Convert.ToInt32(niz[0]);
idn = System.Convert.ToInt32(niz[1]);
// datum = System.Convert.ToDateTime(niz[2]);
datum = DateTime.Now;
interest = System.Convert.ToDouble(niz[3]);
principal = System.Convert.ToDouble(niz[4]);
payment = System.Convert.ToDouble(niz[5]);
newprincipal = System.Convert.ToDouble(niz[6]);
String insert = #"INSERT INTO AmortPlanCoupT(ID, AmortPlanID, CoupDate, Interest, Principal, Payxment, NewPrincipal) VALUES (" + idn + "," + amorplanid + "," + datum + "," + (float)interest + "," + (float)principal + "," + (float)payment + "," + (float)newprincipal + ")";
SqlConnection myconn = new SqlConnection(conn);
// String MyString = #"INSERT INTO Employee(ID, FirstName, LastName) VALUES(2, 'G', 'M')";
try
{
myconn.Open();
SqlCommand cmd = new SqlCommand(insert, myconn);
cmd.ExecuteNonQuery();
myconn.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
label14.Text = niz[0];
}
I have created a Windows console app to test :
I have table test with two columns id (int) , leto (float);
SqlConnection MyConnection = new SqlConnection(#"Data Source=.\SQLEXPRESS;AttachDbFilename=|DataDirectory|\App_Data\Database1.mdf;Integrated Security=True;User Instance=True");
try
{
MyConnection.Open();
String MyString = #"INSERT INTO test(id, leto) VALUES(2, 2)";
SqlCommand MyCmd = new SqlCommand(MyString, MyConnection);
MyCmd.ExecuteScalar();
MyConnection.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
I've been trying different stuff to write data to table, and just can't get them there.
As I've said before on this site - the whole User Instance and AttachDbFileName= approach is flawed - at best! Visual Studio will be copying around the .mdf file and most likely, your INSERT works just fine - but you're just looking at the wrong .mdf file in the end!
If you want to stick with this approach, then try putting a breakpoint on the myConnection.Close() call - and then inspect the .mdf file with SQL Server Mgmt Studio Express - I'm almost certain your data is there.
The real solution in my opinion would be to
install SQL Server Express (and you've already done that anyway)
install SQL Server Management Studio Express
create your database in SSMS Express, give it a logical name (e.g. Database1)
connect to it using its logical database name (given when you create it on the server) - and don't mess around with physical database files and user instances. In that case, your connection string would be something like:
Data Source=.\SQLEXPRESS;Database=Database1;Integrated Security=True
and everything else is exactly the same as before...
Before you go any further, you should rewrite your code to use parameters, instead of string concatenation.