I have an excel sheet which i am uploading to sqlserver table using sqlbulkcopy, i have applied an index on database table so that duplicate entries are ignored but i also want to display the duplicate entries which were ignored in a grid view.
protected void Button1_Click(object sender, EventArgs e)
{
string strFileType = System.IO.Path.GetExtension(FileUpload1.FileName).ToString().ToLower();
string strFileName = FileUpload1.PostedFile.FileName.ToString();
FileUpload1.SaveAs(Server.MapPath("~/Import/" + strFileName + strFileType));
string strNewPath = Server.MapPath("~/Import/" + strFileName + strFileType);
string excelConnectionString = String.Format(#"Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source="+strNewPath +"; Extended Properties=Excel 8.0;");
//string excelConnectionString = String.Format(#"Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source=C:\\myFolder\\Book1.xls;" + "Extended Properties=Excel 8.0;");
// Create Connection to Excel Workbook
using (OleDbConnection connection = new OleDbConnection(excelConnectionString))
{
connection.Open();
OleDbCommand command = new OleDbCommand("Select ID,Data FROM [Sheet1$]", connection);
OleDbCommand command1 = new OleDbCommand("select count(*) from [Sheet1$]",connection);
//Sql Server Table DataTable
DataTable dt4=new DataTable();
SqlCommand cmd4 = new SqlCommand("select id,data from ExcelTable", con);
try
{
SqlDataAdapter da4 = new SqlDataAdapter(cmd4);
da4.Fill(dt4);//sql table datatable
}
catch { }
//excelsheet datatable
DataTable oltlb = new DataTable();
OleDbCommand olcmd = new OleDbCommand("select id,data from [Sheet1$]", connection);
try
{
OleDbDataAdapter olda = new OleDbDataAdapter(olcmd);
olda.Fill(oltlb); //excel table datatable
}
catch { }
var matched = (from table1 in dt4.AsEnumerable()
join table2 in oltlb.AsEnumerable() on
table1.Field<int>("ID") equals table2.Field<int>("ID")
where table1.Field<string>("Data") == table2.Field<string>("Data")
select table1);
DataTable dthi5 = new DataTable();
dthi5 = matched.CopyToDataTable();
GridView1.DataSource = dthi5;
GridView1.DataBind();
GridView1.DataSource = matched.ToList();
ERROR:
Specified cast is not valid.
table1.Field("ID") equals table2.Field("ID")
It would appear that the ID field in one of your tables is not an int.
Rather than one or both of us guessing, try inspecting the data type of the first column in both tables (oltlb and dt4) to see which has the non-integer data type. It may in fact be both.
Here's the thing: even if the value of a string is a numeric literal, you can't cast a string to an int. You'd have to use Int32.Parse or Convert.ToInt32 to convert. So if one of your tables has a string type for the ID column, you would express it like this:
table1.Field<int>("ID") equals Convert.ToInt32(table2.Field<string>("ID"))
If both of the tables have a string ID field, you could just express it this way:
table1.Field<string>("ID") equals table2.Field<string>("ID")
Related
I have two methods, one to insert, update and delete and a second is checking whether data already exists in my database or not. The main purpose of all code is that I don't want to insert duplicate data into the database.
public class DAL : System.Web.UI.Page
{
SqlConnection connection;
SqlCommand cmd;
SqlDataAdapter da;
DataTable dt;
string connectionString = ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString;
public int numofrows;
//Connection Method
public void Connection()
{
connection = new SqlConnection(ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString);
connection.Open();
}
//This Method will Insert,Update and Delete in Database
public void InsertUpdateDelete(string query)
{
////connection = new SqlConnection("Data Source=IM-82B70624D72D;Initial Catalog=AppointmentScheduler;User ID=sa;Password=za3452432760za");
//connection = new SqlConnection(connectionString);
//connection.Open();
this.Connection();
cmd = new SqlCommand(query, connection);
numofrows = cmd.ExecuteNonQuery();
connection.Close();
}
//This Method will read data From Database
public DataTable ReadData(string Query)
{
this.Connection();
da = new SqlDataAdapter(Query, connection);
dt = new DataTable();
da.Fill(dt);
if (dt.Rows.Count > 0)
{
Response.Redirect("Data already Exist");
}
return dt;
}
}
I have to use both above code in index.aspx. How can I use both above code in index.aspx?
I tried to use but that is not working.
index.aspx code:
protected void btnSaveDays_Click(object sender, EventArgs e)
{
this.query = "SELECT DaysName FROM Dayss WHERE Day_Id='" + DropDownListDays.SelectedValue + "'";
dal.ReadData(this.query);
this.query = "INSERT INTO Dayss VALUES ('" + DropDownListDays.SelectedItem.Text + "')";
dal.InsertUpdateDelete(this.query);
Response.Write("Day Inserted Successfully");
}
But this code is not working and generating error
Conversion failed
not Day_Id is int, and Dayss have two columns. One is id and second is
name
Then there is no point to use single quotes with int typed column value. Single quotes is for character column values.
this.query = "SELECT DaysName FROM Dayss WHERE Day_Id = " + DropDownListDays.SelectedValue ;
And if you want to insert just one column, you need specify your column name in your Dayss table like;
this.query = "INSERT INTO Dayss (YourColumnName) VALUES ('" + DropDownListDays.SelectedItem.Text + "')";
Take a look INSERT (Transact-SQL)
I have excel file with 2 tables. I need to read this tables and get all the values from this tables. But all for what I have is:
OleDbConnection cnn = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=D:\MigrateExelSql\Include\TestDb.xlsx; Extended Properties=Excel 12.0;");
OleDbCommand oconn = new OleDbCommand("select * from [Sheet1$]", cnn);
cnn.Open();
OleDbDataAdapter adp = new OleDbDataAdapter(oconn);
DataTable dt = new DataTable();
adp.Fill(dt);
And I don't uderstand what I need to write for get the all values from Username and Email tables. Here is the .xlsx table TestDb Please can somebody help me, because I'm googling the second day and I have no idea for what I must to do.
And when I try to get values by this method it return me an error:
var fileName = string.Format("{0}\\Include\\TestDb.xlsx", Directory.GetCurrentDirectory());
var connectionString = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0}; Extended Properties=Excel 12.0;", fileName);
var adapter = new OleDbDataAdapter("SELECT * FROM [Sheet1$]", connectionString);
var ds = new DataSet();
adapter.Fill(ds, "Username");
var data = ds.Tables["Username"].AsEnumerable();
foreach (var item in data)
{
Console.WriteLine(item);
}
Console.ReadKey();
One more Edit:
string con =
#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=D:\MigrateExelSql\Include\TestDb.xlsx; Extended Properties=Excel 12.0;";
using(OleDbConnection connection = new OleDbConnection(con))
{
connection.Open();
OleDbCommand command = new OleDbCommand("select * from [Sheet1$]", connection);
using(OleDbDataReader dr = command.ExecuteReader())
{
while(dr.Read())
{
var row1Col0 = dr[0];
Console.WriteLine(row1Col0);
}
}
}
Console.ReadKey();
This will read only first column, but when I try to read dr[1] it will return error: Index was outside bound of the array.
Your xlsx file contains only one sheet and in that sheet there is only one column.
A sheet is treated by the OleDb driver like a datatable and each column in a sheet is considered a datacolumn.
You can't read anything apart one table (Sheet1$) and one column (dr[0]).
If you try to read dr[1] then you are referencing the second column and that column doesn't exist in Sheet1.
Just to test, try to add some values in the second column of the Excel file.
Now you can reference dr[1].
My final goal is to retrieve the data from .xlsx file and load them into .mdb (Microsoft Access) file. My code is now like this:
String fileName = "1.xlsx";
OleDbConnection conn = new OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source=" + fileName + ";" + ";Extended Properties=\"Excel 12.0;HDR=YES;IMEX=1\"");
conn.Open();
DataTable dtSheetName = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
string cmdText = "SELECT * FROM [sheet1$A3:B3]";
DataSet ds;
using (OleDbCommand cmd = new OleDbCommand(cmdText))
{
cmd.Connection = conn;
OleDbDataAdapter adpt = new OleDbDataAdapter(cmd);
ds = new DataSet();
adpt.Fill(ds, "sheet1");
}
But this is when I know how many lines are there in the sheet. What if I don't know about this information? Can I somehow manage to know how many lines are there in the sheet?
change the line
string cmdText = "SELECT * FROM [sheet1$A3:B3]";
to
string cmdText = "SELECT * FROM [sheet1$]";
and add the following line to at the end of your code.
int numberOfRows = ds.Tables[0].Rows.Count;
You don't need to assign columns and rows. You cold just do something like:
string cmdText = "SELECt * FROM [sheet1$]"
Or if you want to limit which columns to read:
string cmdText = "SELECt * FROM [sheet1$A1:B100000]"
Edit: Sorry, missed the part about needing to know how many rows there are. cilerler has your answer.
I have Excel file shown bellow
I want to read 1st read only all school names & school address & insert them in SchoolInfo table of mySql database.
After that I want to read data for each school & insert it in StudentsInfo table which has foreign key associated with SchoolInfo table.
I am reading excel sheet something like this.
public static void Import(string fileName)
{
string strConn = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + fileName +
";Extended Properties=\"Excel 12.0;HDR=No;IMEX=1\"";
var output = new DataSet();
using (var conn = new OleDbConnection(strConn))
{
conn.Open();
var dt = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
if (dt != null)
foreach (DataRow row in dt.Rows)
{
string sheet = row["TABLE_NAME"].ToString();
var cmd = new OleDbCommand("SELECT * FROM [+"+sheet+"+]", conn);
cmd.CommandType = CommandType.Text;
OleDbDataAdapter xlAdapter = new OleDbDataAdapter(cmd);
xlAdapter.Fill(output,"School");
}
}
}
Now I am having data in datatable of dataset, Now how do I read desired data & insert it in my sql table.
Try the following steps:
Reading from Excel sheet
First you must create an OleDB connection to the Excel file.
String connectionString = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + path + ";" +
"Extended Properties=Excel 8.0;";
OleDbConnection xlConn = new OleDbConnection(connectionString);
xlConn.Open();
Here path refers to the location of your Excel spreadsheet. E.g. "D:\abc.xls"
Then you have to query your table. For this we must define names for the table and columns first. Click here to find out how.
Now create the command object.
OleDbCommand selectCmd = new OleDbCommand("SELECT * FROM [Sheet1$]", xlConn);
Now we have to store the ouput of the select command into a DataSet using a DataAdapter
OleDbDataAdapter xlAdapter = new OleDbDataAdapter();
objAdapter1.SelectCommand = selectCmd;
DataSet xlDataset = new DataSet();
xlAdapter.Fill(xlDataset, "XLData");
Save the data into variables
Now extract cell data into variables iteratively for the whole table by using
variable = xlDataset.Tables[0].Rows[row_value][column_value].ToString() ;
Write the data from the variables into the MySQL database
Now connect to the MySQL database using an ODBC connection
String mySqlConnectionString = "driver={MySQL ODBC 5.1 Driver};" +
"server=localhost;" + "database=;" + "user=;" + "password=;";
OdbcConnection mySqlConn = new OdbcConnection(mySqlConnectionString);
mySqlConn.Open();
Construct your INSERT statement using the variables in which data has been stored from the Excel sheet.
OdbcCommand mySqlInsert = new OdbcCommand(insertCommand, mySqlConn);
mySqlInsert.ExecuteScalar()
I am importing excel data into sql server using SqlbulkCopy, but the problem is i want to prevent any duplicate records being entered. Is there a way by which duplicate can be ignored or deleted automatically?
protected void Button1_Click(object sender, EventArgs e)
{
string strFileType = System.IO.Path.GetExtension(FileUpload1.FileName).ToString().ToLower();
string strFileName = FileUpload1.PostedFile.FileName.ToString();
FileUpload1.SaveAs(Server.MapPath("~/Import/" + strFileName + strFileType));
string strNewPath = Server.MapPath("~/Import/" + strFileName + strFileType);
string excelConnectionString = String.Format(#"Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source=" + strNewPath + "; Extended Properties=Excel 8.0;");
// Create Connection to Excel Workbook
using (OleDbConnection connection = new OleDbConnection(excelConnectionString))
{
var command = new OleDbCommand("Select ID,Data FROM [Sheet1$]", connection);
connection.Open();
// Create DbDataReader to Data Worksheet
using (DbDataReader dr = command.ExecuteReader())
{
// SQL Server Connection String
string sqlConnectionString = "Data Source=ARBAAZ-1B14C081;Initial Catalog=abc;Integrated Security=True";
con.Open();
DataTable dt1 = new DataTable();
string s = "select count(*) from ExcelTable";
string r = "";
SqlCommand cmd1 = new SqlCommand(s, con);
try
{
SqlDataAdapter da1 = new SqlDataAdapter(cmd1);
da1.Fill(dt1);
}
catch
{
}
int RecordCount;
RecordCount = Convert.ToInt32(cmd1.ExecuteScalar());
r = RecordCount.ToString();
Label1.Text = r;
con.Close();
int prv = Convert.ToInt32(r);
// Bulk Copy to SQL Server
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(sqlConnectionString))
{
bulkCopy.DestinationTableName = "ExcelTable";
SqlBulkCopyColumnMapping mapping1 = new SqlBulkCopyColumnMapping("excelid", "id");
SqlBulkCopyColumnMapping mapping2 = new SqlBulkCopyColumnMapping("exceldata", "data");
bulkCopy.ColumnMappings.Add(mapping1);
bulkCopy.ColumnMappings.Add(mapping2);
bulkCopy.WriteToServer(dr);
}
con.Open();
DataTable dt = new DataTable();
s = "select count(*) from ExcelTable";
r = "";
SqlCommand cmd = new SqlCommand(s, con);
try
{
SqlDataAdapter da = new SqlDataAdapter(cmd);
da.Fill(dt);
}
catch
{
}
RecordCount = Convert.ToInt32(cmd.ExecuteScalar());
r = RecordCount.ToString();
Label1.Text = r;
con.Close();
int ltr = Convert.ToInt32(r);
if (prv == ltr)
{
Label1.Text = "No records Added";
}
else
{
Label1.Text = "Records Added Successfully !";
}
}
}
}
Can this problem be fixed by creating a trigger to delete duplicates? if yes how? i am a newbie i have never created a trigger.
Another problem is no matter what i try i am unable to get column mapping to work. I am unable to upload data when column names in excel sheet and database are different.
You can create INDEX
CREATE UNIQUE INDEX MyIndex ON ExcelTable(id, data) WITH IGNORE_DUP_KEY
And when you will insert data with bulk to db, all duplicate values will not inserted.
no, either you manage it on your dr object on you code before you load it into the db (like running a DISTINCT operation) or you create a trigger on the DB to check. The trigger will reduce the bulk insert's performace though.
Another option would be to bulk insert into a temp table and then insert into your destination table FROM your temp table using a select DISTINCT
as far as I remember,
you could filter out the redundant rows when importing from the excel file itself.
the following SQL query should be used in the OleDbCommand constructor.
var command = new OleDbCommand("Select DISTINCT ID,Data FROM [Sheet1$]", connection);