I am trying a code snippet that add excel data into sql database using SqlBulkCopy. The code snippet is as given below
OleDbConnection connection=null;
string FilePath="";
try
{
if (FileUpload1.HasFile)
{
FileUpload1.SaveAs(Server.MapPath("~/UploadedFolder/"+FileUpload1.FileName));
FilePath = Server.MapPath("~/UploadedFolder/"+FileUpload1.FileName);
}
string path = FilePath;
// Connection String to Excel Workbook
string excelConnectionString = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=Excel 8.0", path);
connection = new OleDbConnection();
connection.ConnectionString = excelConnectionString;
OleDbCommand command = new OleDbCommand("select * from [Sheet1$]", connection);
connection.Open();
// Create DbDataReader to Data Worksheet
DbDataReader dr = command.ExecuteReader();
// SQL Server Connection String
string sqlConnectionString = #"Data Source=sample;Initial Catalog=ExcelImport;User ID=sample;Password=sample";
// Bulk Copy to SQL Server
SqlBulkCopy bulkInsert = new SqlBulkCopy(sqlConnectionString);
bulkInsert.DestinationTableName = "Customer_Table";
bulkInsert.WriteToServer(dr);
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
finally
{
connection.Close();
Array.ForEach(Directory.GetFiles(Server.MapPath("~/UploadedFolder/")), File.Delete);
}
This add the data from excel file to sql server database table. But my requirement is that I need to add the values of excel sheet plus additionally my own values say autogenerated studentid.
So my question is how I will add new values (say studentid, batchcode etc) along with values that read from excel. And these values to be added to each row of excel data.
Example:-
excel contains following columns
CustomerID,City,Country,PostalCode
Now I need to add values to sql server by adding some new columns as
StudentID,CustomerID,BatchCode,City,Country,Email,PostalCode
How can I do it
Please help
You could do the following:
load the excel data into a data table,
Add the remaining columns to the data table,
Set new column values
SqlBulkCopy the data table into SQL Server.
Something like this:
DbDataReader dr = command.ExecuteReader();
DataTable table = new DataTable("Customers");
table.Load(dr);
table.Columns.Add("StudentId", typeof(int));
table.Columns.Add("BatchCode", typeof(string));
table.Columns.Add("Email", typeof(string));
foreach (DataRow row in table.Rows)
{
row["StudentId"] = GetStudentId(row);
row["BatchCode"] = GetBatchCode(row);
row["Email"] = GetEmail(row);
}
// SQL Server Connection String
string sqlConnectionString = #"Data Source=sample;Initial Catalog=ExcelImport;User ID=sample;Password=sample";
// Bulk Copy to SQL Server
SqlBulkCopy bulkInsert = new SqlBulkCopy(sqlConnectionString);
bulkInsert.DestinationTableName = "Customer_Table";
bulkInsert.WriteToServer(table);
Related
Can you help me with the following?
I have two databases: one in SQL Server making the connection with ADO.NET; the other is a DBF database which makes a class for the connection of the same. I want to make a comparison of data from one table to the other, since they have the same structure, and I want to see what data is missing from both the SQL Server and the DBF.
namespace log
{
static class Program
{
private static ECEntities1 dc = new ECEntities1();
public static void Main()
{
// Query in BDD SQL Server
var query =
from HInvoice in dc.HInvoice
where HInvoice.DOB == '2020-03-01'
select HInvoice.DOB
// Print data
foreach (var item in query)
{
Console.WriteLine(item);
}
// Query in BDD BDF
string path = AppDomain.CurrentDomain.BaseDirectory + #"db";
DataTable dt = DBF.ObtenerDatos(path, "GNDITEM.Dbf");
// Print Data
foreach (DataRow dtRow in dt.Rows)
{
// On all tables' columns
foreach (DataColumn dc in dt.Columns)
{
var field1 = dtRow[dc].ToString();
Console.WriteLine(field1);
}
}
}
}
}
I will answer myself, I am new so the question was how could I be focused, but that was not the way so I changed the focus what I did was the following:
I created a new project where I sent the DBF table to my SQL Server database through a stored procedure and from there make the comparison
Create the connection to the Dbf database.
OleDbConnection cnn = new OleDbConnection();
OleDbDataAdapter da = new OleDbDataAdapter();
OleDbCommand cmd;
string PathArchivo = (#"\\ruta_archivo");
string NombreArchivo = "GNDITEM.Dbf";
cnn.ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + #PathArchivo + ";Extended Properties=dBASE IV;User ID=;Password=";
cmd = new OleDbCommand("select * from " + NombreArchivo, cnn);
da.SelectCommand = cmd;
cnn.Close();
var dt = new DataTable();
da.Fill(dt);
Pass the parameters to a DataTable
var dtParametro = new DataTable();
dtParametro.Columns.Add("parametro1", typeof(string));
dtParametro.Columns.Add("parametro2", typeof(string));
dtParametro.Columns.Add("parametro3", typeof(string));
dtParametro.Columns.Add("parametro4", typeof(string));
dtParametro.Rows.Add(drAdd);
// On all tables' columns
Then with a For we fill the DataTable with the information from the DBF table
foreach (DataRow dr in dt.Rows)
{
drAdd["TYPE"] = dr["parametro1"].ToString();
drAdd["EMPLOYEE"] = dr["parametro2"].ToString();
drAdd["CHECK"] = dr["parametro3"].ToString();
drAdd["ITEM"] = dr["parametro4"].ToString();
dtParametro.Rows.Add(drAdd);
}
Finally I connect to the SQL Server database and call the stored procedure
string cadena1 = #"conexionabdd...";
var cmda = new SqlCommand("Insertar_tbl_GNDITEM1", new SqlConnection(cadena));
cmda.CommandType = CommandType.StoredProcedure;
cmda.Parameters.Add("#param", SqlDbType.Structured).Value = dtParametro;
cmda.Connection.Open(); // abrimos la conexion
cmda.ExecuteNonQuery(); // lo ejecutamso
cmda.Connection.Close(); // cerramos la conexion
We review the database and see that the data is already in the same
PS: In addition to this, you must create the database in the SQL Server, also create a datatype for that table and the stored procedure, here is the video that will guide me, I hope it will be helpful for someone else Video
I want to upload Excel data into SQL Server with uploaded by (i.e which pick from session method)
I have created table name, country, address, uploaded by.
Normally the upload is working fine but I want to add the uploaded by value while uploading the excel bulk upload
I don't know how to insert the uploaded by value in bulk upload some one help me
This is my code:
try
{
myExcelConn.Open();
// GET DATA FROM EXCEL SHEET.
OleDbCommand objOleDB = new OleDbCommand("SELECT * FROM [Sheet1$]", myExcelConn);
// READ THE DATA EXTRACTED FROM THE EXCEL FILE.
OleDbDataReader objBulkReader = null;
objBulkReader = objOleDB.ExecuteReader();
// SET THE CONNECTION STRING.
string sCon = sqlconn;
using (SqlConnection con = new SqlConnection(sCon))
{
con.Open();
//SqlConnection p = new SqlConnection(#"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=C:\Users\jetback\source\repos\Projectstudent\webdown_up_load\App_Data\upload.mdf;Integrated Security=True");
// FINALLY, LOAD DATA INTO THE DATABASE TABLE.
oSqlBulk = new SqlBulkCopy(con);
oSqlBulk.DestinationTableName = "Table3"; // TABLE NAME.
oSqlBulk.ColumnMappings.Add("name", "name");
oSqlBulk.ColumnMappings.Add("address", "address");
oSqlBulk.ColumnMappings.Add("country", "country");
oSqlBulk.WriteToServer(objBulkReader);
}
lblConfirm.Text = "DATA IMPORTED SUCCESSFULLY.";
lblConfirm.Attributes.Add("style", "color:green");
}
catch (Exception ex)
{
lblConfirm.Text = ex.Message;
lblConfirm.Attributes.Add("style", "color:red");
}
finally
{
// CLEAR.
oSqlBulk.Close();
oSqlBulk = null;
myExcelConn.Close();
myExcelConn = null;
}
It's better to use DataTable if you want to add more col when there's already data in yr dataSet:
var results = new DataTable();
using(var myExcelConn = new OleDbConnection(excCnnStr))
{
using (var cmd = new OleDbCommand("SELECT * FROM [Sheet1$]", myExcelConn))
{
myExcelConn.Open();
var adapter = new OleDbDataAdapter(cmd);
adapter.Fill(results);
}
}
//add new col
results.Columns.Add("uploadedBy", typeof(System.Int32));
foreach (DataRow row in results.Rows)
{
row["uploadedBy"] = loggedInUserId; // set uploader
}
using (var con = new SqlConnection(sCon))
{
con.Open();
using (var oSqlBulk = new SqlBulkCopy(con))
{
oSqlBulk.DestinationTableName = "Table3";
oSqlBulk.ColumnMappings.Add("name", "name");
oSqlBulk.ColumnMappings.Add("address", "address");
oSqlBulk.ColumnMappings.Add("country", "country");
oSqlBulk.ColumnMappings.Add("uploadedBy", "uploadedBy");
oSqlBulk.WriteToServer(results);
}
}
You will have to get from Excel and then insert into SQL.
//After objBulkReader = objOleDB.ExecuteReader();
if (objBulkReader.HasRows)
{
while (objBulkReader.Read())
{ string newname = objBulkReader[0].ToString();
string address= objBulkReader[1].ToString();
string country= objBulkReader[2].ToString();
string uploadedby=Session["uploader"].ToString();
//insert these into SQL db by SQL connection
}
}
The 0,1 and 2 in objBulkReader are column positions in Excel.
I am reading excel having like million of records first i query my table (no records) and get Datable. i query my table to get columns name as define in my excel sheet using alias.
var dal = new clsConn();
var sqlQuery = "SELECT FETAPE_THEIR_TRANDATE \"Date\" ,ISSUER Issuer, ISSU_BRAN Branch , STAN_NUMB STAN, TERMID TermID, ACQUIRER Acquirer,DEBIT_AMOUNT Debit,CREDIT_AMOUNT Credit,CARD_NUMB \"Card Number\" , DESCRIPTION Description FROM ALLTRANSACTIONS";
var returntable = dal.ReadData(sqlQuery);
DataRow ds = returntable.NewRow();
var dtExcelData = returntable;
So my datatable looks like this,
Then i read records from excel sheet
OleDbConnection con = null;
if (ext == ".xls")
{
con = new OleDbConnection(#"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filepath + ";Extended Properties=Excel 8.0;");
}
else if (ext == ".xlsx")
{
con = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filepath + ";Extended Properties=\"Excel 12.0;IMEX=2;HDR=NO;TypeGuessRows=0;ImportMixedTypes=Text\"");
}
con.Open();
dt = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string getExcelSheetName = dt.Rows[0]["Table_Name"].ToString();
//OleDbCommand ExcelCommand = new OleDbCommand(#"SELECT * FROM [" + getExcelSheetName + #"]", con);
OleDbCommand ExcelCommand = new OleDbCommand("SELECT F1, F2, F3, F4, F5,F6,F7,F8,F9,F10 FROM [Sheet1$]", con);
OleDbDataAdapter ExcelAdapter = new OleDbDataAdapter(ExcelCommand);
try
{
ExcelAdapter.Fill(dtExcelData); //Here I give the datatable which i made previously
}
catch (Exception ex)
{
//lblAlert2.CssClass = "message-error";
//lblAlert2.Text = ex.Message;
}
It reads successfully and fill data in datatable but creating its own column in data table like F1 to F10 how can i move this data to exactly match with my defined columns in datatable
How Will i manage this to not create other columns (f1,f2..f10)
any workaround will be appreciable or Please explain what i am doing wrong and how can i achieve this.
UPDATE :
My Excel file looks like this
The Microsoft.ACE.OLEDB.12.0 driver will handle both types of excel spreadsheets and using the same Extended Properties. i.e. "Excel 12.0" will open both .xls and .xlsx.
Leave the HDR=NO as OLEDB expects them in the first row and they are actually in row 11.
Sadly "TypeGuessRows=0;ImportMixedTypes=Text" is completely ignored by Microsoft.ACE.OLEDB.12.0, you've got to play around with the registry (yuk). Change your IMEX=2 to IMEX=1 to ensure that mixed data types as handled as text.
Change back to using "Select * From [Sheet1$]" and then I'm afriad that you are going to have to handle the source data manually.
OleDbConnection con = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filepath + ";Extended Properties=\"Excel 12.0;IMEX=1;HDR=NO\"");
con.Open();
DataTable dt = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string getExcelSheetName = (string)dt.Rows[0]["Table_Name"];
DataTable xlWorksheet = new DataTable();
xlWorksheet.Load(new OleDbCommand("Select * From [" + getExcelSheetName + "]", con).ExecuteReader());
//More than 11 rows implies 11 header rows and at least 1 data row
if (xlWorksheet.Rows.Count > 11 & xlWorksheet.Columns.Count >= 10)
{
for (int nRow = 11; nRow < xlWorksheet.Rows.Count; nRow++)
{
DataRow returnRow = returntable.NewRow();
for (int nColumn = 0; nColumn < 10; nColumn++)
{
//Note you will probably get conversion problems here that you will have to handle
returnRow[nColumn] = xlWorksheet.Rows[nRow].ItemArray[nColumn];
}
returntable.Rows.Add(returnRow);
}
}
I'm guessing you simply want to add the excel data into your ALLTRANSACTION table? You don't specify but it seems the likely outcome of this. If so this is a terrible way to do it. You don't need to read the whole table into memory append data and then update the database. All you need to do is read the excel file and insert the data to the Oracle table.
Some thoughts, your returntable will contain data so if you just want the structure of the table then add a "Where RowNum=0" to the Select statement. To add the data to your Oracle Database you could 1) Convert to using the Oracle Data Provider (ODP) and then use using OracleBulkCopy Class or 2) simply modify the above to insert row by row as you read the data. As long as you don't have a LOT of data in your Excel spreadsheet it will work just fine. Having said that a Million rows is a LOT so perhaps not the best option. You will need to validate the input as Excel is not the best data source really.
I am using following code to update an Excel file into SQL Server. Code is working but it is not able to insert first row into table.
OleDbConnection OleDb = new OleDbConnection(ConnectionString);
OleDbCommand OleDbCmm = new OleDbCommand(Query,OleDb);
OleDbDataReader OleDbdr;
OleDb.Open();
if (OleDb.State == ConnectionState.Open)
{
OleDbdr = OleDbCmm.ExecuteReader();
SqlBulkCopy BulkCopy = new SqlBulkCopy(ConfigurationManager.ConnectionStrings["connstring"].ToString());
BulkCopy.DestinationTableName = "TempTable";
if (OleDbdr.Read())
{
BulkCopy.WriteToServer(OleDbdr);
}
}
OleDb.Close();
Even I was facing same problem, this is because I was using Read() Method like below.
while (dr.Read()) {
bulkcopy.WriteToServer(dr);
}
Solution to above problem is remove dr.Read() method and while loop
use
bulkcopy.WriteToServer(dr)
without any condition and Read() Method.
One possible reason for this may be that you've indicated in your connection string that first row contains column names (HDR=YES) thus that row is not treated as containing data.
EDIT
Another possible reason for this would be the call to OleDbDataReader.Read() method before passing the reader to SqlBulkCopy object. MSDN states:
The copy operation starts at the next available row in the reader. Most of the time, the reader was just returned by ExecuteReader or a similar call, so the next available row is the first row.
Thus, in your case you should not call OleDbdr.Read() because this advances the reader to the first row; you should let BulkCopy call Read() and it will start reading from the first row.
Your code should be:
OleDbdr = OleDbCmm.ExecuteReader();
SqlBulkCopy BulkCopy = new SqlBulkCopy(ConfigurationManager.ConnectionStrings["connstring"].ToString());
BulkCopy.DestinationTableName = "TempTable";
BulkCopy.WriteToServer(OleDbdr);
You need to set header for the Excel to sql as I have done in my
string conn = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + Your D S+ ";" +
"Extended Properties=Excel 8.0;";
OleDbConnection sSourceConnection = new OleDbConnection(conn);
using (sSourceConnection)
{
DataTable dtExcelData = new DataTable();
string[] SheetNames = GetExcelSheetNames(strFileName);
string[] preColumnHeader = new string[]{ "CarrierId", "StateId", "TerrCd", "ProgramId", "ClassId",
"PremTypeID","Limit50_100", "Limit100_100", "Limit100_200", "Limit300_300", "Limit300_600",
"Limit500_500","Limit500_1mil", "Limit1mil_1mil", "Limit1mil_2mil", "OtherParameter" };
sSourceConnection.Open();
string strQuery = string.Empty;
strQuery = "SELECT * FROM [" + SheetNames[0] + "]";
OleDbDataAdapter oleDA = new OleDbDataAdapter(strQuery, sSourceConnection);
oleDA.Fill(dtExcelData);
sSourceConnection.Close();
string[] colName = new string[dtExcelData.Columns.Count];
int i = 0;
foreach (DataColumn dc in dtExcelData.Columns)
{
colName[i] = dc.ColumnName;
i++;
}
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connStr))
{
bulkCopy.DestinationTableName = "tbl_test";
bulkCopy.WriteToServer(dtExcelData);
}
}
I am using VS2005 C# ASP.NET and SQL Server 2005.
I have a function which import Excel data. I have met a situation when the data inside is inappropriate, it will bring down the SQL Server DB.
E.g. SELECT [Username], [Password] from [userlist$] -> If an excel spreadsheet contains more than [Username] in one column or values below the columns, the server will crash.
E.G.
May I know how can I have a statement to check for this file error before uploading? Prefer if else statements for checking.
Thank you for any help or examples given.
Below is my code snippet for the excel upload:
if (FileImport.HasFile)
{
// Get the name of the Excel spreadsheet to upload.
string strFileName = Server.HtmlEncode(FileImport.FileName);
// Get the extension of the Excel spreadsheet.
string strExtension = Path.GetExtension(strFileName);
// Validate the file extension.
if (strExtension == ".xls" || strExtension == ".xlsx")
{
// Generate the file name to save.
string strUploadFileName = "C:/Documents and Settings/user01/My Documents/Visual Studio 2005/WebSites/MajorProject/UploadFiles/" + DateTime.Now.ToString("yyyyMMddHHmmss") + strExtension;
// Save the Excel spreadsheet on server.
FileImport.SaveAs(strUploadFileName);
// Create Connection to Excel Workbook
string connStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + strUploadFileName + ";Extended Properties=Excel 8.0;";
using (OleDbConnection connection =
new OleDbConnection(connStr))
{
string selectStmt = string.Format("Select [COLUMNS] FROM [userlist$]");
OleDbCommand command = new OleDbCommand(selectStmt, connection);
connection.Open();
Console.WriteLine("Connection Opened");
// Create DbDataReader to Data Worksheet
using (DbDataReader dr = command.ExecuteReader())
{
// SQL Server Connection String
string sqlConnectionString = "Data Source=<datasource>";
// Bulk Copy to SQL Server
using (SqlBulkCopy bulkCopy =
new SqlBulkCopy(sqlConnectionString))
{
bulkCopy.DestinationTableName = "UserDB";
bulkCopy.WriteToServer(dr);
return;
}
}
}
Depending on the amount of data that is in the Excel spreadsheet, you could read each of the values from the column you are interested in into a Dictionary, for example, and fail the import as soon as you find the first conflict.
For example, using your existing datareeader:
Dictionary<string, string> cValues = new Dictionary<string, string>();
// Create DbDataReader to Data Worksheet
using (DbDataReader dr = command.ExecuteReader())
{
while (dr.Read()) {
string sValue = dr[0].ToString();
if (cValue.ContainsKey(sValue)) {
// There is a duplicate value, so bail
throw new Exception("Duplicate value " + sValue);
} else {
cValues.Add(sValue, sValue);
}
}
}
// Now execute the reader on the command again to perform the upload
using (DbDataReader dr = command.ExecuteReader())
You should solve the cause of the problem, which is the database. Apparently, the way you store the data should be via stored procedures which handle wrongly input data.
Also, the constraints on the database tables should prohibit you from storing non-sensical data.