As part of a project I'm working on in C# I need to read in a .dbf file. The first thing I want to do is to get the schema table from the file. I have code that works as long as the filename (without the extension) is not longer than 8 characters.
For example, let's say I have a file named MyLongFilename.dbf. The following code does not work; it throws the following exception: “The Microsoft Jet database engine could not find the object 'MyLongFilename'. Make sure the object exists and that you spell its name and the path name correctly.”
string cxn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyLongFilename;Extended Properties=dBASE 5.0";
OleDbConnection connection = new OleDbConnection(cxn);
To get past this exception, the next step is to use a name the OldDbConnection likes ('MyLongF~1' instead of 'MyLongFilename'), which leads to this:
string cxn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyLongF~1;Extended Properties=dBASE 5.0";
OleDbConnection connection = new OleDbConnection(cxn);
This does successfully return an OleDbConnection. Now to get the schema table I try the following:
connection.Open();
DataTable schemaTable = connection.GetOleDbSchemaTable(OleDbSchemaGuid.Columns,
new object[] { null, null, fileNameNoExt, null });
This returns a DataTable with no rows. If I rename the filename to 8 or less characters then this code works and I get back a row for each field in the database.
With the long filename, I know the returned connection is valid because I can use it to fill a DataSet like so:
string selectQuery = "SELECT * FROM [MyLongF~1#DBF];";
OleDbCommand command = new OleDbCommand(selectQuery, connection);
connection.Open();
OleDbDataAdapter dataAdapter = new OleDbDataAdapter();
dataAdapter.SelectCommand = command;
DataSet dataSet = new DataSet();
dataAdapter.Fill(dataSet);
This gives me back a DataSet containing a DataTable with all of the data from the dbf file.
So the question is how can I get just the schema table for the long named dbf file? Of course I can work around the issue by renaming/copying the file, but that’s a hack I don’t want to have to make. Nor do I want to fill the DataSet with the top 1 record and deduce the schema from columns.
According to MSDN, the folder represents the database and the files represent tables. You should be using the directory path not including the filename in the connection string then, and the name of the table as part of the restrictions to GetOleDbSchemaTable.
Well, i think the connection should be
string cxn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=C:\;Extended Properties=dBASE 5.0";
OleDbConnection connection = new OleDbConnection(cxn);
and the other is, maybe you should try with other provider, I boosted a lot along ago when I used like this:
string cxn = "PROVIDER=VFPOLEDB.1;Data Source=C:\;Extended Properties=dBASE 5.0";
But you should have VFP 7 installed
or install Microsoft OLE DB Provider for Visual FoxPro 9.0 from here
const string connectionString = #"Provider = vfpoledb; Data Source = {0}; Collating Sequence = general;";
OleDbConnection conn = new OleDbConnection(string.Format(connectionString, dirName));
conn.Open();
OleDbCommand cmd = new OleDbCommand(string.Format("select * from {0}", fileName), conn);
Is fileNameNoExt holding the short filename version? Also, MyLongF~1 is 9 characters, not 8.
If you have a single (and possibly small) dbf file you can solve the problem copying the dbf file elsewhere and open the copy instead of the original file.
I believe that the DataSource should represent the directory that contains the .DBF files. Each .DBF file corresponds to a table in that directory.
My guess is c:\MyLongF~1 is a short name for a directory that contains a filename corresponding to MyLongF~1#DBF
Can you verify whether or not this is the case?
Related
Here is my connection string for .txt file and some piece of code
public class FileTransfers
{
public void fileFromDrive(string filename)
{
FileInfo file = new FileInfo(filename);
string fileConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
file.DirectoryName +
"; Extended Properties='text;HDR=YES;FMT=Delimited(,)';";
using (OleDbConnection con = new OleDbConnection(fileConnectionString))
{
using (OleDbCommand cmd = new OleDbCommand(
string.Format("SELECT * FROM [{0}]", file.Name), con))
{
con.Open();
using (OleDbDataAdapter adp = new OleDbDataAdapter(cmd))
{
DataTable tbl = new DataTable("Attendance");
adp.Fill(tbl);
}
}
}
}
}
But the problem is when I debug records in tbl it shows me the data in only one column, but there are 7 multiple columns in my .txt file and hundreds of rows.
I have tried FMT=Delimited(,), FMT=TabDelimited,FMT=FiXed but didn't got multiple columns. I know every entry needs a (,) at is end, but I cant do that manually.
There are some details you need to consider doing this process as Jan Schreuder mentions in his article Using OleDb to Import Text Files,
The Jet engine makes assumptions about the content of the file. This
can result in incorrect imports. For example, it might think a column
contains date values. But in fact, your file should treat the columns
as a string. In these cases, you should create a Schema.Ini file that
describes the type of value for each column. The class creates a
Schema.Ini file before it opens the delimited file, but only to
specify what the delimiter is. You may want to change this to use
pre-defined INI files that describe your input file.
So go ahead and create the schema.ini file as prescribed and you issue will be all gone. It's contents should be looking like this,
[FileName.csv]
ColNameHeader=True
Format=CSVDelimited
For more details on how tos refer to the following MSDN guide,
Schema.ini File (Text File Driver)
I am looking for advice on the best way to parse a Microsoft Excel file and update/store the data into a given SQL Server database. I using ASP.NET MVC so I plan on having a page/view take in an Excel spreadsheet and using that user given file I will need to use C# to parse the data from the columns and update the database based on matches with the spreadsheet column that contains the key column of the database table. The spreadsheet will always be in the same format so I will only need to handle on format. It seems like this could be a pretty common thing I am just looking for the best way to approach this before getting started. I am using Entity Framework in my current application but I don't have to use it.
I found this solution which seems like it could be a good option:
public IEnumerable<MyEntity> ReadEntitiesFromFile( IExcelDataReader reader, string filePath )
{
var myEntities = new List<MyEntity>();
var stream = File.Open( filePath, FileMode.Open, FileAccess.Read );
using ( var reader = ExcelReaderFactory.CreateOpenXmlReader( stream ) )
{
while ( reader.Read() )
{
var myEntity = new MyEntity():
myEntity.MyProperty1 = reader.GetString(1);
myEntity.MyProperty2 = reader.GetInt32(2);
myEntites.Add(myEntity);
}
}
return myEntities;
}
Here is an example of a what a file might look like (Clock# is the key)
So given a file in this format I want to match the user to the data table record using the clock # and update the record with each of the cells information. Each of the columns in the spreadsheet have a relatable column in the data table. All help is much appreciated.
You can use the classes in the namespace Microsoft.Office.Interop.Excel, which abstracts all the solution you found. Instead of me rewriting it, you can check out this article: http://www.codeproject.com/Tips/696864/Working-with-Excel-Using-Csharp.
Better yet, why not bypass the middle man? You can use an existing ETL tool, such as Pentaho, or Talend, or something to go straight from Excel to your database. These types of tools often offer a lot of customization, and are fairly straightforward to use. I've used Pentaho quite a lot for literally what you're describing, and it saved me the head ache of writing the code myself. Unless you want to/need to write it yourself, I think the latter is the best approach.
Try This
public string GetDataTableOfExcel(string file_path)
{
using (OleDbConnection conn = new OleDbConnection())
{
DataTable dt = new DataTable();
string Import_FileName = Server.MapPath(file_path);
//Import_FileName = System.IO.Path.GetDirectoryName(file_path);
string fileExtension = Path.GetExtension(Import_FileName);
if (fileExtension == ".xlsx")
conn.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + Import_FileName + ";" + "Extended Properties='Excel 12.0 Xml;HDR=YES;'";
using (OleDbCommand comm = new OleDbCommand())
{
comm.CommandText = "Select * from [Sheet1$]";
comm.Connection = conn;
using (OleDbDataAdapter da = new OleDbDataAdapter())
{
da.SelectCommand = comm;
da.Fill(dt);
}
}
}
}
Now Your Data in DataTable. You can create insert query from datatable's data.
file_path is excel file's full path with directory name.
I am using following code for excel upload
OleDbConnection sSourceConnection;
string properties = "Excel 8.0; HDR=NO; IMEX=1;";//properties set for connection to excel
string sSourceConstr = #"Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" + filePath + ";Extended Properties=\"" + properties + "\"";
sSourceConnection = new OleDbConnection(sSourceConstr);//creating the OLEDB connection
try
{
//select statement to select data from the first excel sheet
string sql = string.Format("Select * FROM [{0}]", "Sheet1$");
//commands to fill the dataset with excel data
OleDbDataAdapter excelAdapter = new OleDbDataAdapter();
OleDbCommand command = new OleDbCommand(sql, sSourceConnection);
sSourceConnection.Open();
excelAdapter.SelectCommand = command;
excelAdapter.Fill(dSet, EXCEL_DATA);
I have to upload around 300 records. One column has some text comments. The length of comments varies from 10 chars to 1000 chars. But all of comments above 255 length are getting truncated in that column.
I have used this post Excel cell-values are truncated by OLEDB-provider to change a registry setting, but it didn't work.
I have also tried everything mentioned in the post OleDB & mixed Excel datatypes : missing data, still nothing works.
I was using ACE engine so the correct place to update the registery is
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel\TypeGuessRowsHKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel\TypeGuessRows
For Microsoft Office 2010-2013-2016/365
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel
And better is to scan for the text ‘TypeGuessRows’ and when you find it, in combination with Excel, set its value to 0. We’ve found another important location for this behavior at this location
\HKEY_LOCAL_MACHINE \SOFTWARE\Microsoft\Office \ClickToRun\REGISTRY \MACHINE\Software \Wow6432Node \Microsoft \Office\16.0\Access Connectivity Engine\Engines\Excel.
I'm trying to read a pipe separated text file. First lines are
"BewerberID"|"Druck"|"Druckdatum"|"HistorieID"|"Bearbeiter"|"BewZuBewGruppeID"|"Bemerkung"
"12586"|"EinladungOFD.dot "|"03.02.2003 00:00:00"|"162"|"Petersen "|"20295"|"ungültig"
"12807"|"EinladungOFD.dot "|"27.02.2003 00:00:00"|"258"|"Petersen "|"20617"|""
"12807"|"EinladungOFD.dot "|"28.02.2003 00:00:00"|"270"|"Petersen "|"20617"|""
Below is the LINQpad script i'm using. It runs perfectly, but does return values from the first colum only.
string mySelectQuery = "SELECT * FROM Historie.CSV";
OleDbConnection connection = new OleDbConnection
("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\\;" +
"Extended Properties=\"text;HDR=YES;IMEX=1;FMT=Delimited(|)\"");
connection.Open();
OleDbCommand cmd = connection.CreateCommand();
cmd.CommandText = mySelectQuery;
OleDbDataReader rdr = cmd.ExecuteReader();
rdr.Dump();
rdr.Close();
connection.Close();
This returns the first column only.
BewerberID
12586
12807
12807
I tried switching to column names SELECT BewerberID, Druck FROM Historie.CSV but get an error stating "At least one parameter has no value". (BTW: SELECT BewerberID FROM Historie.CSV does work and returns the same as *)
What do i have to do to get all columns back?
Create a file named schema.ini in the same folder as Historie.CSV (in this case C:\). The file should have the following contents:
[Historie.csv]
Format=Delimited(|)
ColNameHeader=True
Then try rerunning the code.
Some links:
When reading a CSV file using a DataReader and the OLEDB Jet data provider, how can I control column data types?
Schema.ini File (Text File Driver)
I know this topic is done to death but I am at wits end.
I need to parse a csv. It's a pretty average CSV and the parsing logic has been written using OleDB by another developer who swore that it work before he went on vacation :)
CSV sample:
Dispatch Date,Master Tape,Master Time Code,Material ID,Channel,Title,Version,Duration,Language,Producer,Edit Date,Packaging,1 st TX,Last TX,Usage,S&P Rating,Comments,Replace,Event TX Date,Alternate Title
,a,b,c,d,e,f,g,h,,i,,j,k,,l,m,,n,
The problem I have is that I get various errors depending on the connection string I try.
when I try the connection string:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source="D:\TEST.csv\";Extended Properties="text;HDR=No;FMT=Delimited"
I get the error:
'D:\TEST.csv' is not a valid path. Make sure that the path name is spelled correctly and that you are connected to the server on which the file resides.
When I try the connection string:
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=D:\TEST.csv;Extended Properties=Excel 12.0;
or the connection string
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=D:\TEST.csv;Extended Properties=Excel 8.0;
I get the error:
External table is not in the expected format.
I am considering throwing away all the code and starting from scratch. Is there something obvious I am doing wrong?
You should indicate only the directory name in your connection string. The file name will be used to query:
var filename = #"c:\work\test.csv";
var connString = string.Format(
#"Provider=Microsoft.Jet.OleDb.4.0; Data Source={0};Extended Properties=""Text;HDR=YES;FMT=Delimited""",
Path.GetDirectoryName(filename)
);
using (var conn = new OleDbConnection(connString))
{
conn.Open();
var query = "SELECT * FROM [" + Path.GetFileName(filename) + "]";
using (var adapter = new OleDbDataAdapter(query, conn))
{
var ds = new DataSet("CSV File");
adapter.Fill(ds);
}
}
And instead of OleDB you could use a decent CSV parser (or another one).
Alternate solution is to use TextFieldParser class (part of .Net framework itself.) https://learn.microsoft.com/en-us/dotnet/api/microsoft.visualbasic.fileio.textfieldparser
This way you do not have to rely on other developer who has gone for holidays. I have used it so many times and have not hit any snag.
I have posted this from work (hence I cannot post an example snippet. I will do so when I go home this evening).
It seems your first row contains the column names, so you need to include the HDR=YES property, like this:
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=D:\TEST.csv;Extended Properties="Excel 12.0;HDR=YES";
Try the connection string:
"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=D:\TEST.csv;Extended Properties=\"Excel 8.0;IMEX=1\""
var s=#"D:\TEST.csv";
string dir = Path.GetDirectoryName(s);
string sConnection = "Provider=Microsoft.Jet.OLEDB.4.0;"
+ "Data Source=\"" + dir + "\\\";"
+ "Extended Properties=\"text;HDR=YES;FMT=Delimited\"";