I am reading excel having like million of records first i query my table (no records) and get Datable. i query my table to get columns name as define in my excel sheet using alias.
var dal = new clsConn();
var sqlQuery = "SELECT FETAPE_THEIR_TRANDATE \"Date\" ,ISSUER Issuer, ISSU_BRAN Branch , STAN_NUMB STAN, TERMID TermID, ACQUIRER Acquirer,DEBIT_AMOUNT Debit,CREDIT_AMOUNT Credit,CARD_NUMB \"Card Number\" , DESCRIPTION Description FROM ALLTRANSACTIONS";
var returntable = dal.ReadData(sqlQuery);
DataRow ds = returntable.NewRow();
var dtExcelData = returntable;
So my datatable looks like this,
Then i read records from excel sheet
OleDbConnection con = null;
if (ext == ".xls")
{
con = new OleDbConnection(#"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filepath + ";Extended Properties=Excel 8.0;");
}
else if (ext == ".xlsx")
{
con = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filepath + ";Extended Properties=\"Excel 12.0;IMEX=2;HDR=NO;TypeGuessRows=0;ImportMixedTypes=Text\"");
}
con.Open();
dt = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string getExcelSheetName = dt.Rows[0]["Table_Name"].ToString();
//OleDbCommand ExcelCommand = new OleDbCommand(#"SELECT * FROM [" + getExcelSheetName + #"]", con);
OleDbCommand ExcelCommand = new OleDbCommand("SELECT F1, F2, F3, F4, F5,F6,F7,F8,F9,F10 FROM [Sheet1$]", con);
OleDbDataAdapter ExcelAdapter = new OleDbDataAdapter(ExcelCommand);
try
{
ExcelAdapter.Fill(dtExcelData); //Here I give the datatable which i made previously
}
catch (Exception ex)
{
//lblAlert2.CssClass = "message-error";
//lblAlert2.Text = ex.Message;
}
It reads successfully and fill data in datatable but creating its own column in data table like F1 to F10 how can i move this data to exactly match with my defined columns in datatable
How Will i manage this to not create other columns (f1,f2..f10)
any workaround will be appreciable or Please explain what i am doing wrong and how can i achieve this.
UPDATE :
My Excel file looks like this
The Microsoft.ACE.OLEDB.12.0 driver will handle both types of excel spreadsheets and using the same Extended Properties. i.e. "Excel 12.0" will open both .xls and .xlsx.
Leave the HDR=NO as OLEDB expects them in the first row and they are actually in row 11.
Sadly "TypeGuessRows=0;ImportMixedTypes=Text" is completely ignored by Microsoft.ACE.OLEDB.12.0, you've got to play around with the registry (yuk). Change your IMEX=2 to IMEX=1 to ensure that mixed data types as handled as text.
Change back to using "Select * From [Sheet1$]" and then I'm afriad that you are going to have to handle the source data manually.
OleDbConnection con = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filepath + ";Extended Properties=\"Excel 12.0;IMEX=1;HDR=NO\"");
con.Open();
DataTable dt = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string getExcelSheetName = (string)dt.Rows[0]["Table_Name"];
DataTable xlWorksheet = new DataTable();
xlWorksheet.Load(new OleDbCommand("Select * From [" + getExcelSheetName + "]", con).ExecuteReader());
//More than 11 rows implies 11 header rows and at least 1 data row
if (xlWorksheet.Rows.Count > 11 & xlWorksheet.Columns.Count >= 10)
{
for (int nRow = 11; nRow < xlWorksheet.Rows.Count; nRow++)
{
DataRow returnRow = returntable.NewRow();
for (int nColumn = 0; nColumn < 10; nColumn++)
{
//Note you will probably get conversion problems here that you will have to handle
returnRow[nColumn] = xlWorksheet.Rows[nRow].ItemArray[nColumn];
}
returntable.Rows.Add(returnRow);
}
}
I'm guessing you simply want to add the excel data into your ALLTRANSACTION table? You don't specify but it seems the likely outcome of this. If so this is a terrible way to do it. You don't need to read the whole table into memory append data and then update the database. All you need to do is read the excel file and insert the data to the Oracle table.
Some thoughts, your returntable will contain data so if you just want the structure of the table then add a "Where RowNum=0" to the Select statement. To add the data to your Oracle Database you could 1) Convert to using the Oracle Data Provider (ODP) and then use using OracleBulkCopy Class or 2) simply modify the above to insert row by row as you read the data. As long as you don't have a LOT of data in your Excel spreadsheet it will work just fine. Having said that a Million rows is a LOT so perhaps not the best option. You will need to validate the input as Excel is not the best data source really.
Related
I'm building a program that needs the following features:
Import an Excel file into database -- Check
Avoid duplicates --- working on it
Ignore some rows that are in the header of the Excel file and the bottom --- that's what I want to ask you guys
Here's my code
protected void Upload_Click(object sender, EventArgs e)
{
string excelPath = Server.MapPath("~/Nova pasta/") + Path.GetFileName(FileUpload1.PostedFile.FileName);
string filepath = Server.MapPath("~/Nova pasta/") + Path.GetFileName(FileUpload1.FileName);
string filename = Path.GetFileName(filepath);
FileUpload1.SaveAs(excelPath);
string strConnection = #"Data Source=PEDRO-PC\SQLEXPRESS;Initial Catalog=costumizado;Persist Security Info=True;User ID=sa;Password=1234";
string excelConnectionString = #"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filepath + ";Extended Properties=\"Excel 12.0 Xml;HDR=NO,IMEX=1;\"";
OleDbConnection excelConnection = new OleDbConnection(excelConnectionString);
excelConnection.Open();
DataTable schema = excelConnection.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string sheetName = schema.Rows[0]["TABLE_NAME"].ToString();
OleDbCommand cmd = new OleDbCommand("Select * from [" + sheetName + "]", excelConnection);
OleDbDataReader dReader;
dReader = cmd.ExecuteReader();
using (SqlBulkCopy sqlBulk = new SqlBulkCopy(strConnection))
{
sqlBulk.ColumnMappings.Add(0,0);
sqlBulk.ColumnMappings.Add(1,1);
sqlBulk.ColumnMappings.Add(2,2);
sqlBulk.ColumnMappings.Add(3,3);
sqlBulk.DestinationTableName = "Dados";
sqlBulk.WriteToServer(dReader);
}
excelConnection.Close();
}
And what I'm struggling is that I need to my code to find the columns in the excel and ignore the rows that I don't need...
I thought that this lines were enough for the job :
sqlBulk.ColumnMappings.Add(0,0);
sqlBulk.ColumnMappings.Add(1,1);
sqlBulk.ColumnMappings.Add(2,2);
sqlBulk.ColumnMappings.Add(3,3);
sqlBulk.DestinationTableName = "Dados";
Here's the table that I want to import :
Dealing with fluctuating source files is tricky. Maybe try loading the file as a whole into a raw table and then after loading the table call a proc to only move the columns that match complete row to your prod or final table. For example, only select from raw where Data Mov, Data Valor, Descricao do Movimento, and Valor em EUR are not null. If needed you could add more validation if needed, like checking the first two columns for date format and the last column for a numeric value. I just think it might be easier to do it in SQL than in .NET code.
I want to read all data from an xls file using OLEDB, but I don't have any experience in that.
string filename = #"C:\Users\sasa\Downloads\user-account-creation_2.xls";
string connString = #"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filename + ";Extended Properties='Excel 8.0;HDR=YES'";
using (System.Data.OleDb.OleDbConnection conn = new System.Data.OleDb.OleDbConnection(connString))
{
conn.Open();
System.Data.OleDb.OleDbCommand selectCommand = new System.Data.OleDb.OleDbCommand("select * from [Sheet1$]", conn);
System.Data.OleDb.OleDbDataAdapter adapter = new System.Data.OleDb.OleDbDataAdapter(selectCommand);
DataTable dt = new DataTable();
adapter.Fill(dt);
int counter = 0;
foreach (DataRow row in dt.Rows)
{
String dataA = row["email"].ToString();
// String dataB= row["DataB"].ToString();
Console.WriteLine(dataA + " = ");
counter++;
if (counter >= 40) break;
}
}
I want to read all data from email row
I get this error
'Sheet$' is not a valid name. Make sure that it does not include invalid characters or punctuation and that it is not too long
Well, you don't have a sheet called Sheet1 do you? Your sheet seems to be called "email address from username" so your query should be....
Select * From ['email address from username$']
Also please don't use Microsoft.Jet.OLEDB.4.0 as it's pretty much obsolete now. Use Microsoft.ACE.OLEDB.12.0. If you specify Excel 12.0 in the extended properties it will open both .xls and .xlsx files.
You can also load the DataTable with a single line...
dt.Load(new System.Data.OleDb.OleDbCommand("Select * From ['email address from username$']", conn).ExecuteReader());
To read the names of the tables in the file use...
DataTable dtTablesList = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
foreach (DataRow drTable in dtTablesList.Rows)
{
//Do Something
//But be careful as this will also return Defined Names. i.e ranges created using the Defined Name functionality
//Actual Sheet names end with $ or $'
if (drTable["Table_Name"].ToString().EndsWith("$") || drTable["Table_Name"].ToString().EndsWith("$'"))
{
Console.WriteLine(drTable["Table_Name"]);
}
}
Is it possible to use the Open XML SDK?
https://learn.microsoft.com/en-us/office/open-xml/how-to-retrieve-the-values-of-cells-in-a-spreadsheet
I am reading data from an Excel file. when I read the normal Excel file,It works fine but when I read an excel file which has columns like shown below it does not find the work sheet and gives an exception-
The Microsoft Jet database engine could not find the object 'Sheet1$_'. Make sure the object exists and that you spell its name and the path name correctly.
My Code to read the excel is-
private static DataTable getExcelData(string ExcelPath)
{
OleDbConnection con;
string connectionString;
string[] pathArray = ExcelPath.Split('.');
var Extention = pathArray[pathArray.Length - 1];
if (Extention == "xlsx")
//read a 2007 file
connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" +
ExcelPath + ";Extended Properties=\"Excel 8.0;HDR=YES;\"";
else
//read a 97-2003 file
connectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
ExcelPath + ";Extended Properties=Excel 8.0;";
con = new OleDbConnection(connectionString);
if (con.State == ConnectionState.Closed)
{
con.Open();
}
DataTable dbSchema = con.GetOleDbSchemaTable(OleDbSchemaGuid.Columns, null);
var firstSheetName = dbSchema.Rows[0]["TABLE_NAME"];
OleDbDataAdapter cmd = new OleDbDataAdapter("select * from [" + firstSheetName + "] Where NOT [Event Code]=''", con);
DataSet ds = new DataSet();
cmd.Fill(ds);
con.Close();
return ds.Tables[0];
}
}
I have to get all the columns inside Mon,Tues etc.
GetOleDbSchemaTable also returns hidden tables in your Excel file: usually a name like Sheet1$_ indicates an hidden table created when you apply a filter on Sheet1$.
You need to change your code: search for table that ends with $ to set firstSheetName.
Please note that OLEDB does not preserve the sheet order as they were in Excel.
Also note that you need to do this to read an excel file with multirow titles:
set HDR=No in EXTENDED PROPERTIES of your connection string
specify column name and select range in your OleDbCommand in order to skip the first two rows
For example:
SELECT [F1] AS Location,
[F2] AS EmpId,
[F3] AS EmpName,
[F4] AS MondayShift,
[F5] AS Monday15Min,
[F6] AS Monday30Min,
[F7] AS Monday15Min2
FROM [Sheet1$A3:G]
The following codes read an excel sheet and copy all the data into a C# DataTable
string strConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + fileName + ";Extended Properties=\"Excel 8.0;HDR=Yes;IMEX=2\"";
conn = new OleDbConnection(strConn);
conn.Open();
string strExcel = "";
OleDbDataAdapter myCommand = null;
DataSet ds = null;
strExcel = "select * from [" + Sheet1$+ "]";
myCommand = new OleDbDataAdapter(strExcel, strConn);
ds = new DataSet();
myCommand.Fill(ds, "table1");
DataTable DT = ds.Tables[0];
When I review the data in DataTable DT, I found that the data in excel which start with apostrophe cannot be read (ex. '0010, '0026, ..., etc), i.e. become empty in DataTable DT.
Any suggested solution to solve it?
This is one of the fun things working with Excel vba.
The only way to see if there is an apostrophe is using this syntax
Dim s As String
Cells(1, 2).Value = "'abc"
s = Cells(1, 2).Formula
s = Cells(1, 2).PrefixCharacter
When you step through this vba code it will only show you the apostrophe if you add the prefixCharacter to your output.
So the only solution I know of is to loop through ALL THE CELLS.... replace the apostrophe with something silly like & #8217; (html apostrophe) and then fix it in your database..
And if you feel like you've been slapped in the face, yes VBA does that to you now and then. This is not OLEDB issue but an excel issue
I'm totally new with OleDB and reading excel files. I have a worksheet with 3 columns (Name - Surname - E-mail Address) and I need to:
know the rows number
read all the addresses in the third columns
extract one by one each address
I use an OpenFileDialog object (ofd) and a TextBox (excel) to display the selected file. This is my code:
if (ofd.ShowDialog() == DialogResult.OK)
{
excel.Text = ofd.FileName;
connection = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + excel.Text + ";Extended Properties=\"Excel 12.0 Xml;HDR=NO;IMEX=1\"";
conn.ConnectionString = connection;
conn.Open();
string name_query = "SELECT A FROM[" + ofd.SafeFileName + "]";
OleDbDataAdapter da = new OleDbDataAdapter(name_query, conn);
da.Fill(table);
conn.Close();
j = table.Rows.Count;
}
It doesn't work, a query problem in the "FROM...". I usually read this type of query:
"SELECT * FROM [Sheet1$]"
but I can't find what Sheet1$ exactly is. Someone could explain me the right query?
2) To access to each element of the table (it would contain only the third column) and save it in a string variable what I have to do?
Thanks a lot!
To get the sheetnames, you can use the default getschema functionality from the data providers (connection.getschema).
Without column headers (HDR=NO), the columns are named F1,F2,etc., so you for the third field, you could query on F3. If you want to be completely sure, you can also use getschema to get the column names of the sheet/table found with the first getschema.
Finally, to get the values in a string list, you can use a bit of Linq (see the stringlist in the example). Not sure if you meant in a single string value, but if that's the case, you can use a string.join on the linq select.
Combined code starting form connection opening:
conn.Open();
var tableschema = conn.GetSchema("Tables");
var firstsheet = tableschema.Rows[0]["TABLE_NAME"].ToString();
string name_query = "SELECT F3 FROM [" + firstsheet + "]";
OleDbDataAdapter da = new OleDbDataAdapter(name_query, conn);
da.Fill(table);
conn.Close();
j = table.Rows.Count;
var stringlist = table.Rows.Cast<DataRow>().Select(dr => dr[0].ToString()).ToList();