C# 2.0 Fastest way to parse Excel spreadsheet [duplicate] - c#

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Reading Excel files from C#
What is the fastest way to read large sets of data from excel from Csharp. Example code would be great . .

In our desktop environment, I have reached the best mix between performance, flexibility and stability by using Excel via COM.
Access to Excel is always via the same thread.
I use late-binding (in VB.Net) to make my app version independent.
The rest of the application is developed in C#, only this part and some other small parts are in VB, because they are easier in VB.Net.
Dim workBook As Object = GetObject(fileName)
Dim workSheet As Object = workBook.WorkSheets.Item(WorkSheetNr)
Dim range As Object = workSheet.Cells.Item(1, 1)
Dim range2 As Object = range.CurrentRegion
Dim rrow As Integer = range2.Row ' For XL97, first convert to integer. XL97 will generate an error '
Dim rcolumn As Integer = range2.Column
Dim top As Object = workSheet.Cells.Item(rrow, rcolumn)
Dim bottom As Object = top.Offset(range2.Rows.Count - 1, range2.Columns.Count - 1)
range = workSheet.Range(top, bottom)
Dim values As Object(,)
values = range.Value
Here you have a 2-dimensional array containing the values from Excel. The last statement gets the data from Excel to .Net.
Since the limits on the size of a Excel sheet, these cannot get very large, so memory should not be a problem.
We have done some tests on performance, on multiple systems. It is optimized to create as few as possible (out-of-process) COM calls.
This way was the one that has given us the best performance, specially since the data is directly in an array, and access to this data is faster as going through a dataset.
Slow in this solution is starting Excel. But if you need to process multiple files, right after each other, the cost of starting Excel is made only once.
Also I would not use this solution in a server environment.

public class ExcelHeaderValues
{
public static string CUSIP = "CUSIP";
public static string ORIG = "ORIG";
public static string PRICE = "PRICE";
public static int COLUMNCOUNT = 3;
}
public class ExcelParser
{
private ArrayList collOutput = null;
string sSheetName = String.Empty;
string[] strValidColumn;
int validRowCnt = 0;
string sColumnPositions = String.Empty;
OleDbCommand ExcelCommand;
OleDbDataAdapter ExcelAdapter;
OleDbConnection ExcelConnection = null;
DataSet dsSheet = null;
string path = string.Empty;
string identifier = string.Empty;
public ExcelParser()
{
collOutput = new ArrayList();
}
public void Extract()
{
bool headermatch = false;
string strCusip = string.Empty, strOrig = string.Empty, strPrice = string.Empty, strData = string.Empty;
string strCusipPos = string.Empty, strPricePos = string.Empty, strOrigPos = string.Empty;
string strColumnHeader = String.Empty;
int reqColcount = 0;
string[] strTemp;
bool bTemp = false;
bool validRow = false;
DataTable schemaTable = GetSchemaTable();
validRowCnt = 0;
foreach (DataRow dr in schemaTable.Rows)
{
if (dsSheet != null)
{
dsSheet.Reset();
dsSheet = null;
}
strCusipPos = string.Empty;
strOrigPos = string.Empty;
strPricePos = string.Empty;
if (isValidSheet(dr))
{
sColumnPositions = string.Empty;
validRowCnt = 0;
foreach (DataRow dataRow in dsSheet.Tables[0].Rows)
{
sColumnPositions = string.Empty;
if (headermatch == false)
{
sColumnPositions = string.Empty;
foreach (DataColumn column in dsSheet.Tables[0].Columns)
{
strColumnHeader = dataRow[column.ColumnName].ToString().ToUpper().Trim();
strColumnHeader = strColumnHeader.ToUpper();
if (strColumnHeader == ExcelHeaderValues.ORIG.ToUpper() || strColumnHeader == ExcelHeaderValues.CUSIP.ToUpper() || strColumnHeader == ExcelHeaderValues.PRICE.ToUpper())
{
bTemp = true;
validRow = true;
reqColcount = ExcelHeaderValues.COLUMNCOUNT;
}
if (bTemp)
{
bTemp = false;
sColumnPositions += column.ColumnName + "^" + strColumnHeader + ";";
}
}
strValidColumn = sColumnPositions.Trim().Split(';');
if (validRow == true && reqColcount == strValidColumn.Length - 1)
{
headermatch = true;
break;
}
validRowCnt++;
}
}
if (headermatch == true)
{
try
{
if (dsSheet.Tables[0].Rows.Count > 0)
{
if (strValidColumn.Length > 0)
{
for (int i = 0; i < strValidColumn.Length - 1; i++)
{
if (strValidColumn[i].ToUpper().Contains("CUSIP"))
{
strTemp = strValidColumn[i].ToString().Split('^');
strCusipPos = strTemp[0].ToString();
strTemp = null;
}
if (strValidColumn[i].ToUpper().Contains("PRICE"))
{
strTemp = strValidColumn[i].ToString().Split('^');
strPricePos = strTemp[0].ToString();
strTemp = null;
}
if (strValidColumn[i].ToUpper().Contains("ORIG"))
{
strTemp = strValidColumn[i].ToString().Split('^');
strOrigPos = strTemp[0].ToString();
strTemp = null;
}
}
}
for (int iData = validRowCnt; iData < dsSheet.Tables[0].Rows.Count; iData++)
{
if (strCusipPos.Trim() != "")
strCusip = dsSheet.Tables[0].Rows[iData][strCusipPos].ToString().Trim();
if (strOrigPos.Trim() != "")
strOrig = dsSheet.Tables[0].Rows[iData][strOrigPos].ToString().Trim();
if (strPricePos.Trim() != "")
strPrice = dsSheet.Tables[0].Rows[iData][strPricePos].ToString().Trim().ToUpper();
strData = "";
if (strCusip.Length == 9 && strCusip != "" && strPrice != "" && strOrig != "" && !strPrice.ToUpper().Contains("SOLD"))
strData = strCusip + "|" + Convert.ToDouble(strOrig) * 1000000 + "|" + strPrice + "|||||";
if (strData != null && strData != "")
collOutput.Add(strData);
strCusip = string.Empty;
strOrig = string.Empty;
strPrice = string.Empty;
strData = string.Empty;
}
}
}
catch (Exception ex)
{
throw ex;
}
}
headermatch = false;
sColumnPositions = string.Empty;
strColumnHeader = string.Empty;
}
}
}
private bool isValidSheet(DataRow dr)
{
bool isValidSheet = false;
sSheetName = dr[2].ToString().ToUpper();
if (!(sSheetName.Contains("$_")) && !(sSheetName.Contains("$'_")) && (!sSheetName.Contains("Print_Titles".ToUpper())) && (dr[3].ToString() == "TABLE" && ((!sSheetName.Contains("Print_Area".ToUpper())))) && !(sSheetName.ToUpper() == "DLOFFERLOOKUP"))
{
if (sSheetName.Trim().ToUpper() != "Disclaimer$".ToUpper() && sSheetName.Trim().ToUpper() != "Summary$".ToUpper() && sSheetName.Trim().ToUpper() != "FORMULAS$" )
{
string sQry = string.Empty;
sQry = "SELECT * FROM [" + sSheetName + "]";
ExcelCommand = new OleDbCommand(sQry, ExcelConnection);
dsSheet = new DataSet();
ExcelAdapter = new OleDbDataAdapter(ExcelCommand);
isValidSheet = false;
try
{
ExcelAdapter.Fill(dsSheet, sSheetName);
isValidSheet = true;
}
catch (Exception ex)
{
isValidSheet = false;
throw new Exception(ex.Message.ToString());
}
finally
{
ExcelAdapter = null;
ExcelCommand = null;
}
}
}
return isValidSheet;
}
private DataTable GetSchemaTable()
{
DataTable dt = null;
string connectionString = String.Empty;
connectionString = GetConnectionString();
ExcelConnection = new OleDbConnection(connectionString);
try
{
ExcelConnection.Open();
dt = ExcelConnection.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
}
catch (Exception ex)
{
throw ex;
}
return dt;
}
private string GetConnectionString()
{
string connStr = String.Empty;
try
{
if (path.ToLower().Contains(".xlsx"))
{
connStr = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source='" + path + "';" + "Extended Properties='Excel 12.0;HDR=No;IMEX=1;'";
}
else if (path.ToLower().Contains(".xlsm"))
{
connStr = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source='" + path + "';" + "Extended Properties='Excel 12.0 Macro;HDR=No;IMEX=1;'";
}
else if (path.ToLower().Contains(".xls"))
{
connStr = "provider=Microsoft.Jet.OLEDB.4.0;Data Source='" + path + "';Extended Properties='Excel 8.0;HDR=No;IMEX=1;'";
}
else
{connStr = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source='" + path + "';" + "Extended Properties='HTML Import;IMEX=1;HDR=No;'";
}
}
catch (Exception ex)
{
throw ex;
}
return connStr;
}
}

Related

Loops in an array for iterate data fields

what should i do with this task, Im so stuck with this.
protected void Button1_Click(object sender, EventArgs e){
string FileExtension = System.IO.Path.GetExtension(FileUpload1.PostedFile.FileName);
if (FileExtension == ".csv" || FileExtension == ".xls" || FileExtension == ".xlsx")
{
string filename = Path.Combine(Server.MapPath("~/UploadExcels"), new Guid() + ".xlsx");
try
{
FileUpload1.PostedFile.SaveAs(filename);
string excelConnectionString = string.Empty;
excelConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" +
filename + ";Extended Properties=\"Excel 12.0;HDR=Yes;IMEX=2\"";
if (FileExtension == ".xls")
{
excelConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
filename + ";Extended Properties=\"Excel 8.0;HDR=Yes;IMEX=2\"";
}
else if (FileExtension == ".xlsx")
{
excelConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" +
filename + ";Extended Properties=\"Excel 12.0;HDR=Yes;IMEX=2\"";
}
string[] Lines = File.ReadAllLines(filename);
string[] Fields;
Lines = Lines.Skip(1).ToArray();
Dictionary<string, Int32> emlist = new Dictionary<string, int>();
List<TaskExport> emList = new List<TaskExport>();
foreach (var line in Lines)
{
Fields = line.Split(new char[] { ';' });
emList.Add(
new TaskExport
{
Plant = Fields[0],
Prod__Number = Fields[1],
Order_Number = Fields[2],
Vehicle_Number = Fields[3],
Delivery_Number = Fields[4],
Vehicle_Type = Fields[5],
Engine_Type = Fields[6],
Plant_Dispatch_Date = Fields[7],
Interior = Fields[8],
Paint = Fields[9],
Model = Fields[10],
Country_Name = Fields[11],
Engine_Number = Fields[12],
Sample_Digit = Fields[13],
Number_Of_Code = Fields[14],
Codes = Fields[15],
Number_Of_FZEG = Fields[215],
Country_Code = Fields[216]
});
using (ExportExcelEntities table = new ExportExcelEntities())
{
foreach (var db in emList)
{
var dt = table.TaskExports.Where(s => s.ID.Equals(db.ID)).FirstOrDefault();
if (dt != null)
{
dt.Plant = db.Plant;
dt.Prod__Number = db.Prod__Number;
dt.Order_Number = db.Prod__Number;
dt.Vehicle_Number = db.Vehicle_Number;
dt.Delivery_Number = db.Delivery_Number;
dt.Vehicle_Type = db.Vehicle_Type;
dt.Engine_Type = db.Engine_Type;
dt.Plant_Dispatch_Date = db.Plant_Dispatch_Date;
dt.Interior = db.Interior;
dt.Paint = db.Paint;
dt.Model = db.Model;
dt.Country_Name = db.Country_Name;
dt.Engine_Number = db.Engine_Number;
dt.Sample_Digit = db.Sample_Digit;
dt.Number_Of_Code = db.Number_Of_Code;
dt.Codes = db.Codes;
dt.Number_Of_FZEG = db.Number_Of_FZEG;
dt.Country_Code = db.Country_Code;
table.TaskExports.Add(db);
}
else
{
table.TaskExports.Add(db);
}
}
table.SaveChanges();
Label1.Text = "Data has been Exported";
}
}
}
catch (Exception ex)
{
Label1.Text = "cant save this data" + ex;
}
}
}
}
}
All i must to do is inserting loop to iterate this "codes = fields[15]"
List<TaskExport> emList = new List<TaskExport>();
foreach (var line in Lines)
{
Fields = line.Split(new char[] { ';' });
emList.Add(
new TaskExport
{
Plant = Fields[0],
Prod__Number = Fields[1],
Order_Number = Fields[2],
Vehicle_Number = Fields[3],
Delivery_Number = Fields[4],
Vehicle_Type = Fields[5],
Engine_Type = Fields[6],
Plant_Dispatch_Date = Fields[7],
Interior = Fields[8],
Paint = Fields[9],
Model = Fields[10],
Country_Name = Fields[11],
Engine_Number = Fields[12],
Sample_Digit = Fields[13],
Number_Of_Code = Fields[14],
Codes = Fields[15],
Number_Of_FZEG = Fields[215],
Country_Code = Fields[216]
});
I have to loop that "codes" field until the fields[15] reach to fields[214]..
somebody please help me with this problem -_-???
First you need to get all attributes of class in order they are declared.
I hope this answer will help you to do so.
Then loop over your property list (assume it's in props variable) and set values.
List<TaskExport> emList = new List<TaskExport>();
foreach (var line in Lines)
{
Fields = line.Split(new char[] { ';' });
TaskExport taskExport = new TaskExport();
for(int i = 0; i < props.Length; i++)
{
prop[i].SetValue(taskExport, Convert.ChangeType(Fields[i], propertyInfo.PropertyType), null);
}
emList.Add(taskExport);
}

How can I get values from a csv file where some of the cells contain commas?

I have a script that imports a csv file and reads each line to update the corresponding item in Sitecore. It works for many of the products but the problem is for some products where certain cells in the row have commas in them (such as the product description).
protected void SubmitButton_Click(object sender, EventArgs e)
{
if (UpdateFile.PostedFile != null)
{
var file = UpdateFile.PostedFile;
// check if valid csv file
message.InnerText = "Updating...";
Sitecore.Context.SetActiveSite("backedbybayer");
_database = Database.GetDatabase("master");
SitecoreContext context = new SitecoreContext(_database);
Item homeNode = context.GetHomeItem<Item>();
var productsItems =
homeNode.Axes.GetDescendants()
.Where(
child =>
child.TemplateID == new ID(TemplateFactory.FindTemplateId<IProductDetailPageItem>()));
try
{
using (StreamReader sr = new StreamReader(file.InputStream))
{
var firstLine = true;
string currentLine;
var productIdIndex = 0;
var industryIdIndex = 0;
var categoryIdIndex = 0;
var pestIdIndex = 0;
var titleIndex = 0;
string title;
string productId;
string categoryIds;
string industryIds;
while ((currentLine = sr.ReadLine()) != null)
{
var data = currentLine.Split(',').ToList();
if (firstLine)
{
// find index of the important columns
productIdIndex = data.IndexOf("ProductId");
industryIdIndex = data.IndexOf("PrimaryIndustryId");
categoryIdIndex = data.IndexOf("PrimaryCategoryId");
titleIndex = data.IndexOf("Title");
firstLine = false;
continue;
}
title = data[titleIndex];
productId = data[productIdIndex];
categoryIds = data[categoryIdIndex];
industryIds = data[industryIdIndex];
var products = productsItems.Where(x => x.DisplayName == title);
foreach (var product in products)
{
product.Editing.BeginEdit();
try
{
product.Fields["Product Id"].Value = productId;
product.Fields["Product Industry Ids"].Value = industryIds;
product.Fields["Category Ids"].Value = categoryIds;
}
finally
{
product.Editing.EndEdit();
}
}
}
}
// when done
message.InnerText = "Complete";
}
catch (Exception ex)
{
message.InnerText = "Error reading file";
}
}
}
The problem is that when a description field has commas, like "Product is an effective, preventative biofungicide," it gets split as well and throws off the index, so categoryIds = data[8] gets the wrong value.
The spreadsheet is data that is provided by our client, so I would rather not require the client to edit the file unless necessary. Is there a way I can handle this in my code? Is there a different way I can read the file that won't split everything by comma?
I suggest use Ado.Net, If the field's data are inside quotes and it will parse it like a field and ignore any commas inside this..
Code Example:
static DataTable GetDataTableFromCsv(string path, bool isFirstRowHeader)
{
string header = isFirstRowHeader ? "Yes" : "No";
string pathOnly = Path.GetDirectoryName(path);
string fileName = Path.GetFileName(path);
string sql = #"SELECT * FROM [" + fileName + "]";
using(OleDbConnection connection = new OleDbConnection(
#"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + pathOnly +
";Extended Properties=\"Text;HDR=" + header + "\""))
using(OleDbCommand command = new OleDbCommand(sql, connection))
using(OleDbDataAdapter adapter = new OleDbDataAdapter(command))
{
DataTable dataTable = new DataTable();
dataTable.Locale = CultureInfo.CurrentCulture;
adapter.Fill(dataTable);
return dataTable;
}
}

Uploading a file to folder

With the code below I am able to save files to folder.
My problem is only two upload fields are mandatory and the remaining three are not. The code works if all the upload fields have a files selected otherswise its throws a NullReferenceException.
if (AnnualReport != null || ProjectReports != null || Publications != null || Other != null || RegistDoc != null) {
int filesize = AnnualReport.PostedFile.ContentLength;
int filesizeP = ProjectReports.PostedFile.ContentLength;
int filesizePub = Publications.PostedFile.ContentLength;
int filesizeOther = Other.PostedFile.ContentLength;
int filesizeReg = RegistDoc.PostedFile.ContentLength;
if (filesize > 2097152 && filesizeP > 2097152 && filesizePub > 1048576 && filesizeOther > 1048576 && filesizeReg > 1048576) {
ScriptManager.RegisterStartupScript(this, this.GetType(), "popup", "alert('Maximum File size For Annual/Project reports is 1.5MB and for the Publications/Other Attachemnets is 1MB');", true);
} else {
const string ReportDirectory = "REPORTS/";
//Other Document
string OtherPath = ReportDirectory + Other.FileName;
string fileNameWithoutExtensionOther = System.IO.Path.GetFileNameWithoutExtension(Other.FileName);
int iterationOther = 1;
while (System.IO.File.Exists(Server.MapPath(OtherPath))) {
OtherPath = string.Concat(ReportDirectory, fileNameWithoutExtensionOther, "-", iterationOther, ".pdf");
iterationOther++;
}
//Registration Document
string RigisDocPath = ReportDirectory + RegistDoc.FileName;
string fileNameWithoutExtensionRegis = System.IO.Path.GetFileNameWithoutExtension(RegistDoc.FileName);
int iterationRE = 1;
while (System.IO.File.Exists(Server.MapPath(RigisDocPath))) {
RigisDocPath = string.Concat(ReportDirectory, fileNameWithoutExtensionRegis, "-", iterationRE, ".pdf");
iterationRE++;
}
//Annual Reports
string ReportPath = ReportDirectory + AnnualReport.FileName;
string fileNameWithoutExtension = System.IO.Path.GetFileNameWithoutExtension(AnnualReport.FileName);
int iteration = 1;
while (System.IO.File.Exists(Server.MapPath(ReportPath))) {
ReportPath = string.Concat(ReportDirectory, fileNameWithoutExtension, "-", iteration, ".pdf");
iteration++;
}
//Project Report
string ProjecttPath = ReportDirectory + ProjectReports.FileName;
string fileNameWithoutExtensionP = System.IO.Path.GetFileNameWithoutExtension(ProjectReports.FileName);
int iterationP = 1;
while (System.IO.File.Exists(Server.MapPath(ProjecttPath))) {
ProjecttPath = string.Concat(ReportDirectory, fileNameWithoutExtensionP, "-", iterationP, ".pdf");
iterationP++;
}
//publication
string publicationPath = ReportDirectory + Publications.FileName;
string fileNameWithoutExtensionPub = System.IO.Path.GetFileNameWithoutExtension(Publications.FileName);
int iterationPub = 1;
while (System.IO.File.Exists(Server.MapPath(publicationPath))) {
publicationPath = string.Concat(ReportDirectory, fileNameWithoutExtensionPub, "-", iterationPub, ".pdf");
iterationPub++;
}
ProjectReports.SaveAs(Server.MapPath(ProjecttPath));
AnnualReport.SaveAs(Server.MapPath(ReportPath));
Publications.SaveAs(Server.MapPath(publicationPath));
RegistDoc.SaveAs(Server.MapPath(RigisDocPath));
Other.SaveAs(Server.MapPath(OtherPath));
The code you posted is very poorly formated. However, the solution to your immediate problem is to move the null checks down to each individual document.
Instead of doing a huge if line (which has questionable logic, as it only checks if ANY of the documents were uploaded)
You can just check if the required documents are present. (looking at your exising code, present means document name object is not null)
If not, throw an error.
If they are, then proceed with the rest of the code, but wrap the individual processing of optional documents in their own null check if-s.
ie.
if (AnnualReport != null) {
//the block that does stuff with the anual report object
}
I did beak down the code into diferent methods like #irreal suggested, like below;
public void PublicationReporting() {
//connection for the datareader
string csoWConn = ConfigurationManager.ConnectionStrings["RegisterCon"].ToString();
SqlConnection csoW_connection = new SqlConnection(csoWConn);
string database = csoW_connection.DataSource.ToString();
csoW_connection.Open();
if (Publications == null)
{
Publications.Dispose();
////
String MyString = #"UPDATE tb_Quadrennial_Report SET PublicationsPath='' WHERE Org_ID = '" + Accrediated_Orgs.SelectedValue + "'";
SqlCommand MyCmd = new SqlCommand(MyString, csoW_connection);
int LastInsertedRecordID;
LastInsertedRecordID = Convert.ToInt32(MyCmd.ExecuteScalar());
}
else
{
int filesizeP = Publications.PostedFile.ContentLength;
if (filesizeP > 2097152)
{
ScriptManager.RegisterStartupScript(this, this.GetType(), "popup", "alert('Maximum File size For Publication is 2.0 MB');", true);
}
else
{
const string ReportDirectory = "REPORTS/";
//publication
string publicationPath = ReportDirectory + Publications.FileName;
string fileNameWithoutExtensionPub = System.IO.Path.GetFileNameWithoutExtension(Publications.FileName);
int iteration = 1; while (System.IO.File.Exists(Server.MapPath(publicationPath)))
{
publicationPath = string.Concat(ReportDirectory, fileNameWithoutExtensionPub, "-", iteration, ".pdf");
iteration++;
}
Publications.SaveAs(Server.MapPath(publicationPath));
String MyString = #"UPDATE tb_Quadrennial_Report SET PublicationsPath='" + publicationPath + "' WHERE Org_ID = '" + Accrediated_Orgs.SelectedValue + "'";
SqlCommand MyCmd = new SqlCommand(MyString, csoW_connection);
int LastInsertedRecordID;
LastInsertedRecordID = Convert.ToInt32(MyCmd.ExecuteScalar());
}
}
}
I then called it o the click event
try{
PublicationReporting();
}
catch (Exception ex)
{
pgError.Text = "Publication Exception Message: " + ex.Message;
}
finally
{
csoW_connection.Close();
}
From here it was pretty easy to figure out the problem.
I just needed to dispose the content in the upload field if no file was selected like this
public void PublicationReporting() {
//connection for the datareader
string csoWConn = ConfigurationManager.ConnectionStrings["RegisterCon"].ToString();
SqlConnection csoW_connection = new SqlConnection(csoWConn);
string database = csoW_connection.DataSource.ToString();
csoW_connection.Open();
if (Publications == null)
{
Publications.Dispose();
////
String MyString = #"UPDATE tb_Quadrennial_Report SET PublicationsPath='' WHERE Org_ID = '" + Accrediated_Orgs.SelectedValue + "'";
SqlCommand MyCmd = new SqlCommand(MyString, csoW_connection);
int LastInsertedRecordID;
LastInsertedRecordID = Convert.ToInt32(MyCmd.ExecuteScalar());
}
else{
//program continues}

Dynamic Crystal Report using Dataset, Microsoft Access Database

I'm trying to get a report from a custom query accessing to a Microsoft Access Database filling a Dataset and then a CrystalReportViwer but I just got an empty report with the column names.
This is my connection:
public String ConectionString = #"Provider=Microsoft.ACE.OLEDB.12.0;Data";
public OleDbConnection miconexion;
public Int32 retorna = 0;
public Int32 conectar(String location){
try
{
ConectionString += #" Source="+location;
miconexion = new OleDbConnection(ConectionString);
//Open Database Conexion
miconexion.Open();
retorna = 1;
}
catch (OleDbException ex){
retorna = 0;
MessageBox.Show("Database Error: "+ex.ToString());
}
return retorna;
}
public void desconectar() {
if(miconexion.State == ConnectionState.Open){
miconexion.Close();
}
}
Method that process the sql //---------------------------------
//txt_CustomConsult is the textbox that contains the custom query.
public String ExecuteSQl()
{
string sql = null;
string inSql = null;
string firstPart = null;
string LastPart = null;
int SelectStart = 0;
int fromStart = 0;
string[] fields = null;
string[] sep = { "," };
int i = 0;
TextObject MyText;
inSql = txt_CustomConsult.Text;
inSql = inSql.ToUpper();
SelectStart = inSql.IndexOf("SELECT");
fromStart = inSql.IndexOf("FROM");
SelectStart = SelectStart + 6;
firstPart = inSql.Substring(SelectStart, (fromStart - SelectStart));
LastPart = inSql.Substring(fromStart, inSql.Length - fromStart);
fields = firstPart.Split(',');
firstPart = "";
for (i = 0; i <= fields.Length - 1; i++) {
if (i > 0)
{
firstPart = firstPart + ", " + fields[i].ToString() + " AS COLUMN" + (i + 1);
firstPart.Trim();
MyText = (TextObject)ObjRep.ReportDefinition.ReportObjects[i + 1];
MyText.Text = fields[i].ToString();
}
else {
firstPart = firstPart + fields[i].ToString() + " AS COLUMN"+(i+1);
firstPart.Trim();
MyText = (TextObject)ObjRep.ReportDefinition.ReportObjects[i + 1];
MyText.Text = fields[i].ToString();
}
}
sql = "SELECT "+firstPart+" "+LastPart;
return sql;
}
// ---------------------------------------------------
Method that exceute report. lstbx_Tablas is the ListBox that contains all tables,
I got all the tables but report doesn't work.
public void DynamicRep() {
try {
//Windows Form where the rpt is located
ReporteResult mirepres = new ReporteResult();
//Connection
conexion miconect = new conexion();
miconect.conectar(Environment.GetEnvironmentVariable("dbUbicacion"));
String sql = ExecuteSQl();
OleDbDataAdapter dscm = new OleDbDataAdapter(sql,miconect.miconexion);
MisConsultas Dataset1 = new MisConsultas();
dscm.Fill(Dataset1,lstbx_Tablas.SelectedItem.ToString());
ObjRep.SetDataSource(Dataset1.Tables[0]);
mirepres.crv_Reporte.ReportSource = ObjRep;
mirepres.crv_Reporte.Refresh();
mirepres.Show();
}
catch (Exception ex) { MessageBox.Show(ex.ToString()); }
}

System.Data.OleDb.OleDbException(0x80004005): Unspecified Error

Through a series of loops and Database calls (Select statements only) saved as datatables I am generating a .txt file with the values. It runs fine for a little while (19 rows...19 full loop cycles) then I get a runtime exception being thrown by connection.Open()
I have the entire loop inside a try catch block in order to catch the exception and produce the message then 2 blank lines then the stack trace.
I have tried to read this and figure out what to do but I am a bit of a Novice when it comes to DB connections. I have looked elsewhere but do not seem to find a question that quite fits my situation.
FYI: C# 4.0 Windows Form Application, Access DB
I am hoping to find some suggestions on where to begin looking. I am positive that my connection is closed when this error is thrown due to the validation i implemented as shown here:
internal IDbConnection GetConnection()
{
try
{
var connection = _assemblyProvider.Factory.CreateConnection();
connection.ConnectionString = _connectionString;
_connectionState = connection.State.ToString();
if (_connectionState == "Open")
GetConnection();
else
{
connection.Open();
}
return connection;
}
catch (Exception exept)
{
throw new Exception(exept.ToString() + "\n\n" + exept.StackTrace.ToString());
}
}
This method is being called from here:
public DataTable ExecuteDataTable(string commandText, string tableName, DbParameterCollection paramCollection, CommandType commandType)
{
DataTable dtReturn;
IDbConnection connection = null;
try
{
connection = _connectionManager.GetConnection();
dtReturn = _dbAdapterManager.GetDataTable(commandText, paramCollection, connection, tableName, commandType);
}
finally
{
if (connection != null)
{
connection.Close();
connection.Dispose();
}
}
return dtReturn;
}
public DataTable ExecuteDataTable(string commandText, string tableName, CommandType commandType)
{
return ExecuteDataTable(commandText, tableName, new DbParameterCollection(), commandType);
}
public DataTable ExecuteDataTable(string commandText)
{
return ExecuteDataTable(commandText, string.Empty, CommandType.Text);
}
and
//read from DB using a SQL statement and return a DataTable
internal static DataTable readDB(string SQL)
{
var dbHelper = new DbHelper();
using (IDbConnection connection = dbHelper.GetConnObject())
{
return dbHelper.ExecuteDataTable(SQL);
}
}
Here is the loop (its kinda long and could probably be done better but I just want to find why its breaking after its worked several times)
The exception is thrown from the line that Reads:
DataTable iRecNum2ClaimRecNumFromClaim = dbConnect.readDB(SQLString);
inside this:
SQLString = "SELECT * FROM Claim WHERE ClaimStatus <> 1";
DataTable allRecsFromClaimNotStatus1 = dbConnect.readDB(SQLString);
if (allRecsFromClaimNotStatus1.Rows.Count == 0)
return;
else
{
string path = txtExtractFileLocation.Text;
if (txtExtractFileLocation.Text.Substring(txtExtractFileLocation.Text.Length - 2) == "\\\\")
{
path = path.Substring(0, path.Length - 1);
}
if (path.Substring(path.Length - 1) == "\\")
path += "DI_Extract.txt";
else
path += #"\DI_Extract.txt";
using (StreamWriter sw = new StreamWriter(#path))
{
for (int i = 0; i < allRecsFromClaimNotStatus1.Rows.Count; i++)
{
rNum = allRecsFromClaimNotStatus1.Rows[i][2].ToString().Trim();//Claim.InsuredRecNum
SQLString = "SELECT * FROM Insured WHERE RecNum = " + rNum;
DataTable allInsuredByRecNum = dbConnect.readDB(SQLString);
lossDate = allRecsFromClaimNotStatus1.Rows[i][11].ToString().Trim();//Claim.LossDate
lossDate = (Convert.ToDateTime(lossDate)).Date.ToString("MM/dd/yyyy");
reportedDate = allRecsFromClaimNotStatus1.Rows[i][9].ToString().Trim();//Claim.ReportedDate
reportedDate = (Convert.ToDateTime(reportedDate)).Date.ToString("MM/dd/yyyy");
claim = allRecsFromClaimNotStatus1.Rows[i][0].ToString().Trim();//Claim.ClaimNumber
if (chkIncludePaymentsForCurrentMonth.Checked == true)
{
direct = allRecsFromClaimNotStatus1.Rows[i][4].ToString().Trim();//Claim.DirectReserve
WP = allRecsFromClaimNotStatus1.Rows[i][5].ToString().Trim();//Claim.WPReserve
ceded = allRecsFromClaimNotStatus1.Rows[i][6].ToString().Trim();//Claim.CededReserve
}
else
{
direct = allRecsFromClaimNotStatus1.Rows[i][29].ToString().Trim();//Claim.MonthEndDirect
WP = allRecsFromClaimNotStatus1.Rows[i][30].ToString().Trim();//Claim.MonthEndWP
ceded = allRecsFromClaimNotStatus1.Rows[i][31].ToString().Trim();//Claim.MonthEndCeded
}
ced = Convert.ToDecimal(ceded);
wav = Convert.ToDecimal(WP);
ceded = ced.ToString("#.##");
WP = wav.ToString("#.##");
if (ceded == "")
ceded = "0";
if (WP == "")
WP = "0";
if ((allRecsFromClaimNotStatus1.Rows[i][10].ToString().Trim() != null) &&
(allRecsFromClaimNotStatus1.Rows[i][10].ToString().Trim() != ""))//Claim.WaiverDate
{
onWaiver = "YES";
}
else
{
onWaiver = "NO";
}
reinsPreNotice = "NO";
reinsCeded = "NO";
switch (allRecsFromClaimNotStatus1.Rows[i][7].ToString().Trim())//Claim.CededPre
{
case "1":
{
reinsPreNotice = "YES";
break;
}
case "2":
{
reinsCeded = "YES";
break;
}
}//end switch
state = allRecsFromClaimNotStatus1.Rows[i][8].ToString().Trim();//Claim.LossState
lName = allInsuredByRecNum.Rows[0][1].ToString().Trim();//Insured.LastName
fName = allInsuredByRecNum.Rows[0][0].ToString().Trim();//Insured.FirstName
mi = allInsuredByRecNum.Rows[0][2].ToString().Trim();//Insured.MI
policy = allInsuredByRecNum.Rows[0][43].ToString().Trim();//Insured.PolicyNumber
DOB = allInsuredByRecNum.Rows[0][10].ToString().Trim();//Insured.DOB
DOB = (Convert.ToDateTime(DOB)).Date.ToString("MM/dd/yyyy");
age = allInsuredByRecNum.Rows[0][11].ToString().Trim();//Insured.TrueAge
issueAge = calculateAge(Convert.ToDateTime(allInsuredByRecNum.Rows[0][10].ToString().Trim()), //Insured.DOB
Convert.ToDateTime(allInsuredByRecNum.Rows[0][45].ToString().Trim()));//Insured.EffectiveDate
SQLString = "SELECT InsuredRecNum, RecNum FROM Claim WHERE InsuredRecNum = " + rNum;
DataTable iRecNum2ClaimRecNumFromClaim = dbConnect.readDB(SQLString);
rNum = iRecNum2ClaimRecNumFromClaim.Rows[0][1].ToString().Trim();
issueDate = allInsuredByRecNum.Rows[0][45].ToString().Trim();//Insured.EffectiveDate
issueDate = (Convert.ToDateTime(issueDate)).Date.ToString("MM/dd/yyyy");
sex = allInsuredByRecNum.Rows[0][13].ToString().Trim();//Insured.Gender
planCode = allInsuredByRecNum.Rows[0][44].ToString().Trim();//Insured.PlanMnemonic
issueAmt = allInsuredByRecNum.Rows[0][49].ToString().Trim();//Insured.BenefitAmount (Monthly Benefit Amount before Offset)
benefitPeriod = allInsuredByRecNum.Rows[0][50].ToString().Trim();//Insured.BenefitPeriod
if (allInsuredByRecNum.Rows[0][54].ToString().Trim().Length == 2)//Insured.EliminationPeriod
eliminationPeriod = "0" + allInsuredByRecNum.Rows[0][54].ToString().Trim();
else
eliminationPeriod = allInsuredByRecNum.Rows[0][54].ToString().Trim();
premiumAmount = allInsuredByRecNum.Rows[0][48].ToString().Trim();//Insured.AnnualPremium
occupationClass = allInsuredByRecNum.Rows[0][55].ToString().Trim();//Insured.OccupationClass
//select only status = EXEC (0)
SQLString = "SELECT * FROM Offset WHERE ClaimRecNum = " + rNum + " AND Status = 0";
DataTable allOffsetByClaimRecNumAndStatus0 = dbConnect.readDB(SQLString);
offsetAmt = 0;
dblSTDOffsetAmount = 0;
dblRecOffsetAmount = 0;
RECOffsetOcc = "0";
RECOffsetExecuted = "0";
int offsetCount = allOffsetByClaimRecNumAndStatus0.Rows.Count;
if (offsetCount != 0)
{
for (int j = 0; j < offsetCount; j++)
{
//accumulate standard offset (STD) and Recovery offset (REC)
if (allOffsetByClaimRecNumAndStatus0.Rows[0][1].ToString().Trim() == "0")//Offset.Type
{
//Standard Type
dblSTDOffsetAmount += Convert.ToDouble(allOffsetByClaimRecNumAndStatus0.Rows[j][4].ToString().Trim());//Offset.Amount
}
else
{
//Recovery type
dblRecOffsetAmount = Convert.ToDouble(allOffsetByClaimRecNumAndStatus0.Rows[j][4].ToString().Trim());//Offset.Amount
RECOffsetOcc = allOffsetByClaimRecNumAndStatus0.Rows[j][5].ToString().Trim();//Offset.Occurance
RECOffsetExecuted = allOffsetByClaimRecNumAndStatus0.Rows[j][6].ToString().Trim();//Offset.Executed
}//end if
}//end for loop
}//end if
STDOffsetAmount = dblSTDOffsetAmount.ToString();
RECOffsetAmount = dblRecOffsetAmount.ToString();
if (chkIncludePaymentsForCurrentMonth.Checked == true)
SQLString = "SELECT * FROM Payment WHERE InsuredRecNum = " + rNum + " AND IssueDate >= #01/01/" + DateTime.Today.Date.Year + "# AND IssueDate <= #" + DateTime.Today.Date.ToShortDateString() + "#";
else
SQLString = "SELECT * FROM Payment WHERE InsuredRecNum = " + rNum + " AND IssueDate >= #01/01/" + endDate.Substring(endDate.Length - 4) + "# AND IssueDate <= #" + Convert.ToDateTime(endDate).Date.ToShortDateString() + "#";
DataTable allPaymentByIRecNumAndIssDateInRange = dbConnect.readDB(SQLString);
YTDPmt = 0;
if (allPaymentByIRecNumAndIssDateInRange.Rows.Count == 0)
YTDPmt = 0;
else
{
int paymentCount = allPaymentByIRecNumAndIssDateInRange.Rows.Count;
double issAmt;
for (int k = 0; k < paymentCount; k++)
{
issAmt = Convert.ToDouble(allPaymentByIRecNumAndIssDateInRange.Rows[0][30].ToString().Trim());//Payment.IssueAmount
YTDPmt += issAmt;
}// end loop
}//end if
YTDPmts = YTDPmt.ToString();
if (chkIncludePaymentsForCurrentMonth.Checked == true)
SQLString = "SELECT * FROM Payment WHERE ClaimRecNum = " + rNum;
else
SQLString = "SELECT * FROM Payment WHERE ClaimRecNum = " + rNum + " AND IssueDate <= #" + Convert.ToDateTime(endDate).Date.ToShortDateString() + "#";
DataTable allPaymentByRNum = dbConnect.readDB(SQLString);
totalPmt = 0;
if (allPaymentByRNum.Rows.Count == 0)
totalPmt = 0;
else
{
double issAmt = Convert.ToDouble(allPaymentByRNum.Rows[0][30].ToString().Trim());
for (int m = 0; m < allPaymentByRNum.Rows.Count; m++)
{
totalPmt += issAmt;
}
}
allPmts = totalPmt.ToString();
//set spacing for output
string block1 = policy + claim + planCode;
block1 = setSpacing(block1, 28);
string block2 = setSpacing(benefitPeriod, 3) + eliminationPeriod + occupationClass;
block2 = setSpacing(block2, 11);
issueAmt = setSpacing(issueAmt, 8);
STDOffsetAmount = setSpacing(STDOffsetAmount, 8);
RECOffsetAmount = setSpacing(RECOffsetAmount, 8);
RECOffsetOcc = setSpacing(RECOffsetOcc, 3);
RECOffsetExecuted = setSpacing(RECOffsetExecuted, 3);
string block3 = lossDate + age;
block3 = setSpacing(block3, 13);
issueAge = setSpacing(issueAge, 3);
string block4 = issueDate + DOB + sex + onWaiver + premiumAmount;
block4 = setSpacing(block4, 32);
reinsPreNotice = setSpacing(reinsPreNotice, 3);
reinsCeded = setSpacing(reinsCeded, 4);
double ap = Convert.ToDouble(allPmts);
allPmts = ap.ToString("#.#");
allPmts = setSpacing(allPmts, 8);
YTDPmts = setSpacing(YTDPmts, 8);
lName = setSpacing(lName, 19);
fName = fName + " " + mi;
fName = setSpacing(fName, 20);
string block5 = state + direct;
block5 = setSpacing(block5, 10);
ceded = setSpacing(ceded, 8);
WP = setSpacing(WP, 8);
reportedDate = setSpacing(reportedDate, 10);
//save row data for text file
dataOutput = (block1 + block2 + issueAmt + STDOffsetAmount + RECOffsetAmount + RECOffsetOcc + RECOffsetExecuted +
block3 + issueAge + block4 + reinsPreNotice + reinsCeded + allPmts + YTDPmts + lName + fName +
block5 + ceded + WP + reportedDate);
//Write to the output record DI_Extract.txt
sw.WriteLine(dataOutput);
counter++;
pbrRecordsProcessed.Value = counter;
}//end for loop
}//end streamwriter
}//end if
After looking deeper into the code I realized that the connection was trying to open 3 times before it closed. Not sure why I was not getting exceptions all the time but correcting this issue not only sped up the application tremendously it cleared the exception.

Categories

Resources