I need to read and write dbf files. Microsoft dropped support for this in Office 2013.
I am trying to use OLEDB. The problem I am having at this point (have not gone into the writing) is that when I read the header info I am having it returned sorted alphabetically while the rows are not sorted like that. I need to have them saved in the same order so that I can go back and create a dbf files (after some data processing) to pass back to my legacy applications which require DB.
This is for internal distribution so I can take care of having the correct .NET library if necessary. I can't find any reference that solves this problem and I rather use a .NET technology than writing dbf from scratch...
I apologize if this ha been answered before is this is the case I'd appreciate being pointed to the correct way of accomplishing this task.
Code to follow
if (intype == 6) //dbase
{
int rowCount = 0;
int colCount = 0;
string npath = filein;
i = npath.LastIndexOf("\\");
aux2 = MySubStr(npath, i + 1, 1); // directory --- internal library
auxstr = MySubStr(npath, i + 1, 2);
i = auxstr.IndexOf(".");
if (i > -1)
auxstr = MySubStr(auxstr, i, 1);
DataSet ds = new DataSet();
OleDbConnection connection;
string connstr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source = '"; // I can change this to proper library
connstr = string.Concat(connstr, aux2);
connstr = string.Concat(connstr, "';");
connstr = string.Concat(connstr, "Extended Properties='dBASE 5.0';"); // again not worried about DBASE 3 or 4
connection = new OleDbConnection(connstr);
try
{
connection.Open();
DataTable dbSchema = connection.GetOleDbSchemaTable(OleDbSchemaGuid.Columns, new object[] { null, null, auxstr, null });
i = 0;
foreach (DataRow rownm in dbSchema.Rows)
{
header[i] = rownm["COLUMN_NAME"].ToString();
myDT[i++] = rownm["DATA_TYPE"].ToString(); // not using at this point... will be used when writing dbfs
}
colCount = i;
aux2 = "Select * from ";
aux2 = string.Concat(aux2, auxstr);
OleDbDataAdapter da = new OleDbDataAdapter(aux2, connection);
da.Fill(ds,"ZDATA");
count = ds.Tables[0].Rows.Count;
}
catch (Exception e)
{
auxstr = e.ToString();
messages[cntmsgs++] = auxstr;
zcode = 99;
logerror(messages, cntmsgs, zcode);
return 99;
}
System.Data.OleDb.OleDbCommand cmd1 = new System.Data.OleDb.OleDbCommand(aux2, connection);
OleDbDataReader reader;
reader = cmd1.ExecuteReader();
StreamWriter ftmp = new StreamWriter(fileout[0], false, System.Text.Encoding.Default);
while (reader.Read())
{
if (rowCount == 0)
{
auxstr ="";
for (i = 0; i < colCount -1; i++)
{
auxstr += "\"" + header[i] + "\",";
}
auxstr += "\"" + header[colCount - 1] + "\"";
ftmp.WriteLine(auxstr);
rowCount++;
}
aux2 = "";
for (i = 0; i < reader.FieldCount; i++)
{
if (reader.IsDBNull(i))
auxstr = "";
else
auxstr = reader.GetString(i);
auxstr = "\"" + auxstr + "\"";
if (i < reader.FieldCount - 1)
{
auxstr = string.Concat(auxstr, ",");
aux2 = string.Concat(aux2, auxstr);
}
else
{
aux2 = string.Concat(aux2, auxstr);
}
}
ftmp.WriteLine(aux2);
}
ftmp.Close();
reader.Close();
connection.Close();
filein = fileout[0];
intype = 2; // right now forcing additional processing in another block...
return 0;
}
You mention .dbf files, but are they really dBASE 5 or was that just a connection you were able to get. I personally have utilized Microsoft's Visual Foxpro OleDb provider. You can make a connection, query the tables, build parameterized select, insert, update, delete and don't need to really know about header and such.
There are many questions about connections to dbf tables and such, but I normally use an OleDbDataAdapter, put my command to execute in that and pull the data via .Fill() into a DataTable object.
Then I can do things like
foreach( DataColumn dc in MyTable.Columns )
[write whatever output of things like dc.ColumnName, dc.Type, etc]
foreach( DataRow dr in MyTable.Rows )
{
write out... dr["WhateverColumn"]
}
No, not exact code, but simple to work with. Building commands to push data back is easy too. But if your big deal is writing the content out... Are you really trying to rewrite the table and reconverting everything to strings?
The other comment about too much noise is accurate... In summary, yes, Microsoft dropped support for whatever, but what do you WANT to do.
Related
I've been developing a C# WPF project with VS2015 using SQL Server Express LocalDb with Entity Framework. I have built a custom seeder for the database, that reads test data from an Excel file, that simply combines the Excel data into a command string, and this is inserted using context.Database.ExecuteSQLCommand.
Now, I was thinking of launching the project with SQL Server Compact Edition 4.0, but I find this command is not working anymore. Do I have to write my uploader again using SqlCeConnection and SqlCeCommand or am I missing something?
Also, from somewhere I have understood that with EF you can switch the SQL provider and the code would not need other changes. Am I in for more surprises down the road?
Example of the uploader command:
string cmd = "INSERT INTO Venues(Name, City, Telephone) Values ('X','Y','Z')"
context.Database.ExecuteSqlCommand(cmd);
The error:
There was an error parsing the query. [ Token line number = 2,Token line offset = 1,Token in error = INSERT ]
This is not just a testing issue, as I would want to include this uploader in the production version, too, for quick inserting of master data (e.g. employee list).
EDIT: Uploader code. If this can be done without resorting to raw SQL, that would be a good solution, too.
This loops through Excel sheets (named after entities) and columns (first row has property name) and rows 2->n (data). This handles the upload of basically any amount of data within Excel limitations. The point is that the code has no knowledge of the entities (might have been possible to parameterize DataContext too). Code might not be optimal, as I'm just a beginner, but has worked for me, except not with SQL CE. Editing to suit CE is not a big issue, but I wanted to ask for possibly better ways.
public static class ExcelUploader
{
static ArrayList data;
static List<string> tableNames;
public static string Upload(string filePath)
{
string result = "";
data = new ArrayList();
tableNames = new List<string>();
ArrayList upLoadData = ReadFile(filePath);
List<string> dataList = ArrayListToStringList(upLoadData);
using (var db = new DataContext())
{
using (var trans = db.Database.BeginTransaction())
{
try
{
foreach (var cmd in dataList)
{
Console.WriteLine(cmd);
db.Database.ExecuteSqlCommand(cmd);
}
db.SaveChanges();
trans.Commit();
}
catch (Exception e)
{
trans.Rollback();
result = e.Message;
MessageBox.Show(result);
}
}
}
return result;
}
private static ArrayList ReadFile(string fileName)
{
List<string> commands = new List<string>();
var xlApp = new Microsoft.Office.Interop.Excel.Application();
var wb = xlApp.Workbooks.Open(fileName, ReadOnly: true);
xlApp.Visible = false;
foreach (Worksheet ws in wb.Worksheets)
{
var r = ws.UsedRange;
var array = r.Value;
data.Add(array);
tableNames.Add(ws.Name);
}
wb.Close(SaveChanges: false);
xlApp.Quit();
return data;
}
private static List<string> ArrayListToStringList(ArrayList arrList)
{
List<string> result = new List<string>();
for(int tableAmount = 0;tableAmount<data.Count;tableAmount++)
{
result.Add(ArrayToSqlCommand(arrList[tableAmount] as Array, tableNames[tableAmount]));
}
return result;
}
private static string ArrayToSqlCommand(Array arr, string tableName)
{
int propertyRow = 1;
int firstDataRow = 2;
string command = "";
// loop rows
for (int rowIndex = firstDataRow; rowIndex <= arr.GetUpperBound(0); rowIndex++)
{
command += "INSERT INTO " + tableName + "(";
//add column names
for (int colIndex = 1; colIndex <= arr.GetUpperBound(1); colIndex++)
{
//get property name
command += arr.GetValue(propertyRow, colIndex);
//add comma if not last column, otherwise close bracket
if (colIndex == arr.GetUpperBound(1))
{
command += ") Values (";
}
else
{
command += ", ";
}
}
//add values
for (int colIndex = 1; colIndex <= arr.GetUpperBound(1); colIndex++)
{
//get property value
command += "'" + arr.GetValue(rowIndex, colIndex) + "'";
//add comma if not last column, otherwise close bracket
if (colIndex == arr.GetUpperBound(1))
{
command += ");";
}
else
{
command += ", ";
}
}
command += "\n";
}
return command;
}
}
There are two ways to use raw SQL queries I'd offer.
Initial data
1) Excel table
+=======+=======+===========+
| Name | City | Telephone |
|===========================|
| Adam | Addr1 | 111-11-11 |
|-------|-------|-----------|
| Peter | Addr2 | 222-22-22 |
+-------+-------+-----------+
2) SQL Server CE table
CREATE TABLE Venues
(
Id int identity primary key,
[Name] nvarchar(100) null,
City nvarchar(100) null,
Telephone nvarchar(100) null
);
3) Getting data from Excel
Here we're interested in getting array from Excel sheet. As soon as we get it, we can safely close Excel. The code assumes file "Employees.xlsx" to be next to executable file.
private object[,] GetExcelData()
{
xlApp = new Excel.Application { Visible = false };
var xlBook =
xlApp.Workbooks.Open(System.IO.Path.Combine(
Environment.CurrentDirectory,
"Employees.xlsx"));
var xlSheet = xlBook.Sheets[1] as Excel.Worksheet;
// For process termination
var xlHwnd = new IntPtr(xlApp.Hwnd);
var xlProc = Process.GetProcesses()
.Where(p => p.MainWindowHandle == xlHwnd)
.First();
// Get Excel data: it's 2-D array with lower bounds as 1.
object[,] arr = xlSheet.Range["A1"].CurrentRegion.Value;
// Shutdown Excel
xlBook.Close();
xlApp.Quit();
xlProc.Kill();
GC.Collect();
GC.WaitForFullGCComplete();
return arr;
}
Now you can use one of the ways to generate query.
Option 1. Use ExecuteSqlCommand
When using ExecuteSqlCommand, it's advisable to use parameterized queries to avoid errors. You can pass explicitly created SqlCeParameter or just pass a value.
private void UseExecuteSqlCommand()
{
object[,] arr = GetExcelData();
using (var db = new EmpContext())
{
db.Database.Initialize(true);
int count = 0;
string sql = "INSERT INTO Venues (Name, City, Telephone) " +
"VALUES (#name, #city, #phone);";
// Start from 2-nd row since we need to skip header
for (int r = 2; r <= arr.GetUpperBound(0); ++r)
{
db.Database.ExecuteSqlCommand(
sql,
new SqlCeParameter("#name", (string)arr[r, 1]),
new SqlCeParameter("#city", (string)arr[r, 2]),
new SqlCeParameter("#phone", (string)arr[r, 3])
);
++count;
}
conn.Close();
MessageBox.Show($"{count} records were saved.");
}
}
Option 2. Use DbConnection
If you want your code to be more generic, you can create method which would accept DbConnection. This will allow to pass either SqlConnection or SqlCeConnection. But the code becomes more verbose because we can't use constructors since these classes are abstract.
private void UseDbConnection()
{
object[,] arr = GetExcelData();
using (var db = new EmpContext())
{
db.Database.Initialize(true);
int count = 0;
string sql = "INSERT INTO Venues (Name, City, Telephone) " +
"VALUES (#name, #city, #phone);";
DbParameter param = null;
DbConnection conn = db.Database.Connection;
conn.Open();
DbCommand command = conn.CreateCommand();
command.CommandText = sql;
command.CommandType = CommandType.Text;
// Create parameters
// Name
param = command.CreateParameter();
param.ParameterName = "#name";
command.Parameters.Add(param);
// City
param = command.CreateParameter();
param.ParameterName = "#city";
command.Parameters.Add(param);
// Telephone
param = command.CreateParameter();
param.ParameterName = "#phone";
command.Parameters.Add(param);
// Start from 2-nd row since we need to skip header
for (int r = 2; r <= arr.GetUpperBound(0); ++r)
{
command.Parameters["#name"].Value = (string)arr[r, 1];
command.Parameters["#city"].Value = (string)arr[r, 2];
command.Parameters["#phone"].Value = (string)arr[r, 3];
command.ExecuteNonQuery();
++count;
}
conn.Close();
MessageBox.Show($"{count} records were saved.");
}
}
You can also use ordinal positions for parameters which eliminates creating parameters names and makes code much shorter:
private void UseDbConnection()
{
object[,] arr = GetExcelData();
using (var db = new EmpContext())
{
db.Database.Initialize(true);
int count = 0;
// Take a note - use '?' as parameters
string sql = "INSERT INTO Venues (Name, City, Telephone) " +
"VALUES (?, ?, ?);";
DbConnection conn = db.Database.Connection;
conn.Open();
DbCommand command = conn.CreateCommand();
command.CommandText = sql;
command.CommandType = CommandType.Text;
// Create parameters
command.Parameters.Add(command.CreateParameter());
command.Parameters.Add(command.CreateParameter());
command.Parameters.Add(command.CreateParameter());
for (int r = 2; r <= arr.GetUpperBound(0); ++r)
{
// Access parameters by position
command.Parameters[0].Value = (string)arr[r, 1];
command.Parameters[1].Value = (string)arr[r, 2];
command.Parameters[2].Value = (string)arr[r, 3];
command.ExecuteNonQuery();
++count;
}
conn.Close();
MessageBox.Show($"{count} records were saved.");
}
}
P.S.
I didn't check whether the underlying connection is opened, but it's a good idea to do so.
Based on JohnyL's excellent input, I was able to modify my code so that it works with either SQL Server Express and and SQL Server CE. I'll put my new code as an answer, as I had to parameterize it further, as I couldn't write the property names in the code either. But this was a simple step, once I got the idea from JohnyL. Not sure though, if the database writing operation should be wrapped inside a DbTransaction, but this worked for now.
public static class ExcelUploader
{
static ArrayList data;
static List<string> tableNames;
static List<DbCommand> cmdList = new List<DbCommand>();
static DbConnection conn;
public static void Upload(string filePath)
{
data = new ArrayList();
tableNames = new List<string>();
//get Excel data to array list
ArrayList upLoadData = ReadFile(filePath);
using (var db = new DataContext())
{
conn = db.Database.Connection;
//transform arraylist into a list of DbCommands
ArrayListToCommandList(upLoadData);
conn.Open();
try
{
foreach (var cmd in cmdList)
{
//Console.WriteLine(cmd.CommandText);
cmd.ExecuteNonQuery();
}
}
catch (Exception e)
{
var result = e.Message;
MessageBox.Show(result);
}
}
}
//opens Excel file and reads worksheets to arraylist
private static ArrayList ReadFile(string fileName)
{
List<string> commands = new List<string>();
var xlApp = new Microsoft.Office.Interop.Excel.Application();
var wb = xlApp.Workbooks.Open(fileName, ReadOnly: true);
xlApp.Visible = false;
foreach (Worksheet ws in wb.Worksheets)
{
var r = ws.UsedRange;
var array = r.Value;
data.Add(array);
tableNames.Add(ws.Name);
}
wb.Close(SaveChanges: false);
xlApp.Quit();
return data;
}
//transforms arraylist to a list of DbCommands
private static void ArrayListToCommandList(ArrayList arrList)
{
List<DbCommand> result = new List<DbCommand>();
for (int tableAmount = 0; tableAmount < data.Count; tableAmount++)
{
ArrayToSqlCommands(arrList[tableAmount] as Array, tableNames[tableAmount]);
}
}
private static void ArrayToSqlCommands(Array arr, string tableName)
{
//Excel row which holds property names
int propertyRow = 1;
//First Excel row with values
int firstDataRow = 2;
string sql = "";
DbCommand cmd = conn.CreateCommand();
sql += "INSERT INTO " + tableName + "(";
//add column names to command text
for (int colIndex = 1; colIndex <= arr.GetUpperBound(1); colIndex++)
{
//get property name
sql += arr.GetValue(propertyRow, colIndex);
//add comma if not last column, otherwise close bracket
if (colIndex == arr.GetUpperBound(1))
{
sql += ") Values (";
}
else
{
sql += ", ";
}
}
//add value parameter names to command text
for (int colIndex = 1; colIndex <= arr.GetUpperBound(1); colIndex++)
{
//get property name
sql += "#" + arr.GetValue(propertyRow, colIndex);
//add comma if not last column, otherwise close bracket
if (colIndex == arr.GetUpperBound(1))
{
sql += ");";
}
else
{
sql += ", ";
}
}
//add data elements as command parameter values
for (int rowIndex = firstDataRow; rowIndex <= arr.GetUpperBound(0); rowIndex++)
{
//initialize command
cmd = conn.CreateCommand();
cmd.CommandText = sql;
cmd.CommandType = CommandType.Text;
for (int colIndex = 1; colIndex <= arr.GetUpperBound(1); colIndex++)
{
//set parameter values
DbParameter param = null;
param = cmd.CreateParameter();
param.ParameterName = "#" + (string)arr.GetValue(propertyRow, colIndex);
cmd.Parameters.Add(param);
cmd.Parameters[param.ParameterName].Value = arr.GetValue(rowIndex, colIndex);
}
//add command to command list
cmdList.Add(cmd);
}
}
}
I've written some code that copies the data from an Azure database into an Excel file. This can be found at the end of this question.
The problem is it takes forever to populate an excel sheet when I have 10k rows for one of the tables. Obviously, this is not ideal for Excel but at this point it has to be done this way. I'm wondering if there is a faster way to code this.
Certainly, creating excel sheet is the bottleneck, because C# grabs the dataset in seconds. If I go into Excel and view the data and then right click and copy with headers and paste that into and excel sheet it also does this in seconds.
So can I programmatically do this?
private void createExcelFile()
{
string fileName = "FvGReport.xlsx";
string filePath = HttpContext.Current.Request.MapPath("~/App_Data/" + fileName); //check www.dotnetperls.com/mappath
string sqlQuery = "";
List<string> sheetNames = new List<string>();
foreach (ListItem item in ddlSummary_Supplier.Items)
{
string sqlSummary = "SELECT * FROM FvGSummaryAll WHERE Supplier_Code = '" + item.Text + "'; ";
sqlQuery = sqlQuery + sqlSummary;
sheetNames.Add("Summary " + item.Text);
string sqlPaymentsSummary = "SELECT * FROM FvGSummaryPayment WHERE Supplier_Code = '" + item.Text + "'; ";
sqlQuery = sqlQuery + sqlPaymentsSummary;
sheetNames.Add("PaymentSummary " + item.Text);
}
DataSet dataSet = new DataSet();
//string sqlQuery = #"SELECT * FROM FvGData WHERE Supplier_Code = 'SFF Pacific'; SELECT * FROM FvGSummaryPayment";
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlDataAdapter adapter = new SqlDataAdapter();
adapter.SelectCommand = new SqlCommand(sqlQuery, connection);
adapter.Fill(dataSet);
}
//this reference conflicts with System.Data as both have DataTable. So defining it here.
Microsoft.Office.Interop.Excel.Application ExcelApp = new Microsoft.Office.Interop.Excel.Application();
Microsoft.Office.Interop.Excel.Workbook excelWorkBook = null;
Microsoft.Office.Interop.Excel.Worksheet excelWorkSheet = null;
ExcelApp.Visible = true;
excelWorkBook = ExcelApp.Workbooks.Add(Microsoft.Office.Interop.Excel.XlWBATemplate.xlWBATWorksheet);
//excel rows start at 1 not 0
try
{
for (int i = 1; i < dataSet.Tables.Count; i++)
{
excelWorkBook.Worksheets.Add(); //Adds new sheet in Excel WorkBook
}
for (int i = 0; i < dataSet.Tables.Count; i++)
{
int dsRow = 1;
excelWorkSheet = excelWorkBook.Worksheets[i + 1];
//Writing Columns Name in Excel Sheet
for (int col = 1; col < dataSet.Tables[i].Columns.Count; col++)
{
excelWorkSheet.Cells[dsRow, col] = dataSet.Tables[i].Columns[col - 1].ColumnName;
}
dsRow++;
for (int xlRow = 0; xlRow < dataSet.Tables[i].Rows.Count; xlRow++)
{
//Excel row and col positions for writing row = 1, col = 1
for (int col = 1; col < dataSet.Tables[i].Columns.Count; col++)
{
excelWorkSheet.Cells[dsRow, col] = dataSet.Tables[i].Rows[xlRow][col - 1].ToString();
}
dsRow++;
}
excelWorkSheet.Name = sheetNames[i]; //Renaming ExcelSheets
}
excelWorkBook.SaveAs(filePath);
excelWorkBook.Close();
ExcelApp.Quit();
Marshal.ReleaseComObject(excelWorkSheet);
Marshal.ReleaseComObject(excelWorkBook);
Marshal.ReleaseComObject(ExcelApp);
}
catch (Exception ex)
{
lblNoData.Text = ex.ToString();
}
finally
{
foreach (Process process in Process.GetProcessesByName("Excel"))
{
process.Kill();
}
}
downloadExcel(filePath, fileName);
}
It looks like you're using Office Automation, which is usually slow on things like this, in my experience. I would suggest saving the output as a delimited file (.csv) and using automation to open that file (or files) with Excel and then save it as a spreadsheet.
I would suggest you to try using some ETL tool, Esspecially if you will do this from time to time again. If you take Talend, for instance, .... you connect to the DB and the schema will be pulled on its own. Take a SQL input component and connect it to a Excel component and you are done. Take about 5 minutes, without a single line of code
I'm not sure what you mean by 'forever', but for comparison I have a process that writes an OpenXML Spreadsheet of 46,124 row with ~500 characters per row in less than 17-seconds. This is generated on by a C# process that is on a separate server at the same hosting facility as the database server.
If writing to a CSV is an option, then that is going to be the solution with the best performance. OpenXML will give you the next best performance, I found the following article to be the most helpful when I was trying to put together my process:
Read-and-Write-Microsoft-Excel-with-Open-XML-SDK
Regarding memory -- You have two things you need to put in memory, your incoming data and your outgoing file. Regardless of what type of file you write, you'll want to use a SqlDataReader instead of your dataSet. This means your incoming data will only have one row at a time in memory (instead of all 10K). When writing your file (CSV or OpenXML) if you write directly to disk (FileStream) instead of memory (MemoryStream) you'll only have that little bit in memory.
Especially if you are running code that is running within your website, you don't want to use up a bunch of memory at once because .NET/IIS doesn't handle that very well.
I have had a problem for a few days and nothing online seems to do it.
I have an SQL table that has 150 columns. I am reading data from an ODBC connection and I want to insert that data into the SQL table. Basically duplicate the ODBC table as SQL.
My problem is that if I put everything in a string and insert it I face a hell of a time with escape characters and exceptions that I can't figure out.
Is there a way to parametrize my insert values that doesn't involve me naming each and every on of them separatly.
This is what I have right now. Also, if anyone knows an easier way to move an ODBC table to an SQL form please let me know
private void fillTable(string tableName)
{
String query = "SELECT * FROM " + tableName;
OdbcCommand command = new OdbcCommand(query, Program.myConnection);
OdbcDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess);
int columnCount = reader.FieldCount;
for (int i = 0; i < columnCount; i++)
{
SqlCommand sCommand = new SqlCommand("ALTER TABLE " + tableName + " ADD " + reader.GetName(i) + " varchar(MAX)", Program.myConnection2);
sCommand.ExecuteNonQuery();
}
string row="";
while (!reader.IsClosed)
{
try
{
row = "";
reader.Read();
for (int i = 0; i < columnCount; i++)
{
if (reader != null)
{
if (reader.GetString(i).Contains('\''))
{
Console.WriteLine("REPLACED QUOT");
String s = reader.GetString(i).Replace("\'", "A");
Console.WriteLine(s);
row += s;
}
else
{
row += "\'" + reader.GetString(i).Trim() + "\',";
}
// Console.WriteLine("FILLER: " + reader.GetString(i));
}
//Console.WriteLine(row);
}
//Console.WriteLine();
row = row.Substring(0, row.Length - 1);
SqlCommand insertCommand = new SqlCommand("INSERT INTO " + tableName + " VALUES(\'1\'," + row + ")", Program.myConnection2);
insertCommand.ExecuteNonQuery();
}
catch (SqlException exp)
{
Console.WriteLine(exp.StackTrace);
Console.WriteLine(row);
// this.Close();
}
}
Program.myConnection2.Close();
}
You can write a method that creats parameter names automatically, adds it to command, and returns the name so that you can use it in the query:
private int _paramCounter = 1;
private string CreateParameter(SqlCommand command, object value) {
string name = "#P" + _paramCounter.ToString();
_paramCounter++;
command.Parameters.AddWithValue(name, value);
return name;
}
Usage:
row += CreateParameter(insertCommand, reader.GetString(i).Trim()) + ",";
Note that you need to create the command object before you loop through the columns. Also, although not needed, you might want to reset the _paramCounter for each row, otherwise the parameter names get longer in the end.
I have just written what has to be considered utterly hideous code to count the rows that contain data in the worksheets called "Data" from all the spreadsheets in a given directory. Here's the code
private const string _ExcelLogDirectoryPath = #"..\..\..\..\Model\ExcelLogs\";
static void Main()
{
var excelLogPaths = Directory.GetFiles(_ExcelLogDirectoryPath, "*.xl*");
var excel = new Microsoft.Office.Interop.Excel.Application();
var excelRowCounts = new Dictionary<string, int>();
foreach (var filePath in excelLogPaths)
{
var spreadsheet = excel.Workbooks.Open(Path.GetDirectoryName(System.Windows.Forms.Application.ExecutablePath) + "/" + filePath);
var worksheet = spreadsheet.Sheets["Data"] as Worksheet;
if (worksheet != null)
{
// var rowCount = UsedRange.Rows.Count - 1; DOES NOT WORK, THE number is bigger than the 'real' answer
var rowCount = 0;
for (var i = 1 ; i < 1000000000; i++)
{
var cell = worksheet.Cells[i, 1].Value2; // "Value2", great name for a property, thanks guys
if (cell != null && cell.ToString() != "") // Very fragile (e.g. skipped rows will break this)
{
rowCount++;
}
else
{
break;
}
}
var name = spreadsheet.Name.Substring(spreadsheet.Name.IndexOf('p'), spreadsheet.Name.IndexOf('.') - spreadsheet.Name.IndexOf('p'));
excelRowCounts.Add(name, rowCount - 1);
}
}
I cannot believe this is the right way to do this. It is crazy slow and includes calls to properties with names like Value2 that do not feel like an intended part of a public API. But the method suggested elsewhere dramatically over reports the number of rows (with data in them).
What is the correct was to count the rows with data in them from an Excel worksheet?
========== EDIT 1 ==========
The reason that both UsedRange.Rows.Count and Sid's ACE.OLEDB solution over report the number of rows appears to be a pink background colour that is applied to some of the columns (but only extending to row 7091). Is there a simple/elegant way to count the rows with data in them (i.e. with non-null cell values) regardless of the display colour?
========== EDIT 2 ===========
Sid's ACE.OLEDB solution with the addition he suggests so that the tSQL line reads
var sql = "SELECT COUNT (*) FROM [" + sheetName + "$] WHERE NOT F1 IS NULL";
works. I'll mark that as the answer.
This should do the trick. You can call it with each filename to retrieve the number of rows.
private string GetNumberOfRows(string filename, string sheetName)
{
string connectionString;
string count = "";
if (filename.EndsWith(".xlsx"))
{
connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filename + ";Mode=ReadWrite;Extended Properties=\"Excel 12.0;HDR=NO\"";
}
else if (filename.EndsWith(".xls"))
{
connectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filename + ";Mode=ReadWrite;Extended Properties=\"Excel 8.0;HDR=NO;\"";
}
string SQL = "SELECT COUNT (*) FROM [" + sheetName + "$]";
using (OleDbConnection conn = new OleDbConnection(connectionString))
{
conn.Open();
using (OleDbCommand cmd = new OleDbCommand(SQL, conn))
{
using (OleDbDataReader reader = cmd.ExecuteReader())
{
reader.Read();
count = reader[0].ToString();
}
}
conn.Close();
}
return count;
}
There might be an even faster way of retrieving just the row count, but I know this works.
if you use interop is why don't use UsedRange?
_Worksheet.UsedRange.Rows.Count
A project I'm working on contains an MDB (acecss database) file. I'd like to export the contents of the tables to text, but am having a hard time finding a way to do it easily using C#. Is there a faster way than using OLEDB and queries?
Update:
Ideally I'd like to not have to statically name each table (there are hundreds) and I have to use .NET 2.0 or below.
There might be a more efficient way, but you could populate the data into a DataTable, and then export to a text file:
Getting data into the DataTable:
string connString = "Provider=Microsoft.ACE.OLEDB.12.0;data source=C:\\marcelo.accdb";
DataTable results = new DataTable();
using(OleDbConnection conn = new OleDbConnection(connString))
{
OleDbCommand cmd = new OleDbCommand("SELECT * FROM Clientes", conn);
conn.Open();
OleDbDataAdapter adapter = new OleDbDataAdapter(cmd);
adapter.Fill(results);
}
Exporting the DataTable to CSV:
EDIT I haven't tested this, but something like this should work for .NET 2.0.
//initialize the strinbuilder
StringBuilder sb = new StringBuilder();
//append the columns to the header row
string[] columns = new string[dt.Columns.Count - 1];
for (int i = 0; i < dt.Columns.Count; i++)
columns[i] = dt.Columns[i].ColumnName;
sb.AppendLine(string.Join(",", columns));
foreach (DataRow row in dt.Rows)
{
//append the data for each row in the table
string[] fields = new string[row.ItemArray.Length];
for (int x = 0; x < myDataRow.ItemArray.Length; x++)
arr[x] = row[x].ToString();
sb.AppendLine(string.Join(",", fields));
}
File.WriteAllText("test.csv", sb.ToString());
No obvious way comes to mind. Just write something that iterates through the tables and spits out the data in whatever text format you want (.csv, tab delimited, etc).
You could always write it in VBA inside of Access, but I don't know if that would make it faster or slower.
If you want to go the Interop route, you can do it in a single command with the Access TransferText method:
using Access = Microsoft.Office.Interop.Access;
using System.Runtime.InteropServices;
static void ExportToCsv(string databasePath, string tableName, string csvFile) {
Access.Application app = new Access.Application();
app.OpenCurrentDatabase(databasePath);
Access.DoCmd doCmd = app.DoCmd;
doCmd.TransferText(Access.AcTextTransferType.acExportDelim, Type.Missing, tableName, csvFile, true);
app.CloseCurrentDatabase();
Marshal.FinalReleaseComObject(doCmd);
doCmd = null;
app.Quit();
Marshal.FinalReleaseComObject(app);
app = null;
}
I do not know C#, but here is another idea, but quite rough. It uses Microsoft.Office.Interop.Access.Dao
DBEngine dbEng = new DBEngine();
Workspace ws = dbEng.CreateWorkspace("", "admin", "",
WorkspaceTypeEnum.dbUseJet);
Database db = ws.OpenDatabase("z:\\docs\\test.accdb", false, false, "");
foreach (TableDef tdf in db.TableDefs)
{
string tablename=tdf.Name;
if (tablename.Substring(0,4) != "MSys")
{
string sSQL = "SELECT * INTO [Text;FMT=Delimited;HDR=Yes;DATABASE=Z:\\Docs].[out_"
+ tablename + ".csv] FROM " + tablename;
db.Execute(sSQL);
}
}