I'm trying to use the OleDb CSV parser to load some data from a CSV file and insert it into a SQLite database, but I get an exception with the OleDbAdapter.Fill method and it's frustrating:
An unhandled exception of type
'System.Data.ConstraintException'
occurred in System.Data.dll
Additional information: Failed to
enable constraints. One or more rows
contain values violating non-null,
unique, or foreign-key constraints.
Here is the source code:
public void InsertData(String csvFileName, String tableName)
{
String dir = Path.GetDirectoryName(csvFileName);
String name = Path.GetFileName(csvFileName);
using (OleDbConnection conn =
new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
dir + #";Extended Properties=""Text;HDR=No;FMT=Delimited"""))
{
conn.Open();
using (OleDbDataAdapter adapter = new OleDbDataAdapter("SELECT * FROM " + name, conn))
{
QuoteDataSet ds = new QuoteDataSet();
adapter.Fill(ds, tableName); // <-- Exception here
InsertData(ds, tableName); // <-- Inserts the data into the my SQLite db
}
}
}
class Program
{
static void Main(string[] args)
{
SQLiteDatabase target = new SQLiteDatabase();
string csvFileName = "D:\\Innovations\\Finch\\dev\\DataFeed\\YahooTagsInfo.csv";
string tableName = "Tags";
target.InsertData(csvFileName, tableName);
Console.ReadKey();
}
}
The "YahooTagsInfo.csv" file looks like this:
tagId,tagName,description,colName,dataType,realTime
1,s,Symbol,symbol,VARCHAR,FALSE
2,c8,After Hours Change,afterhours,DOUBLE,TRUE
3,g3,Annualized Gain,annualizedGain,DOUBLE,FALSE
4,a,Ask,ask,DOUBLE,FALSE
5,a5,Ask Size,askSize,DOUBLE,FALSE
6,a2,Average Daily Volume,avgDailyVolume,DOUBLE,FALSE
7,b,Bid,bid,DOUBLE,FALSE
8,b6,Bid Size,bidSize,DOUBLE,FALSE
9,b4,Book Value,bookValue,DOUBLE,FALSE
I've tried the following:
Removing the first line in the CSV file so it doesn't confuse it for real data.
Changing the TRUE/FALSE realTime flag to 1/0.
I've tried 1 and 2 together (i.e. removed the first line and changed the flag).
None of these things helped...
One constraint is that the tagId is supposed to be unique. Here is what the table look like in design view:
Can anybody help me figure out what is the problem here?
Update:
I changed the HDR property from HDR=No to HDR=Yes and now it doesn't give me an exception:
OleDbConnection conn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + dir + #";Extended Properties=""Text;HDR=Yes;FMT=Delimited""");
I assumed that if HDR=No and I removed the header (i.e. first line), then it should work... strangely it didn't work. In any case, now I'm no longer getting the exception.
The new problem arose here: SQLiteDataAdapter Update method returning 0
First thing: IMO emplacing keys and constraints on a transport structure such as your dataset is a bad idea.
Use a transaction and let the target db throw if it wants.
Second: have you examined the ds to ensure the csv is getting loaded? VS has a dataset debug visualizer built in - simply set a break point after the ds is loaded and hover over the variable name, click the little down-arrow and select the appropriate visualizer.
Third: I don't think that you are generating an insert command. Just before you call update, check the InsertCommand.CommandText..
var cmdText = sqliteAdapter.InsertCommand.CommandText;
I think you will find that it is empty.
This is the source of the SQLiteDataAdapter ctor that ultimately gets called. Note that no command builder is employed. You need to explicitly set the InserCommand property on the SQLiteDataAdapter, perhaps by using a SQLiteCommandBuilder?
public SQLiteDataAdapter(string commandText, SQLiteConnection connection)
{
this.SelectCommand = new SQLiteCommand(commandText, connection);
}
I changed the HDR property from HDR=No to HDR=Yes and now it doesn't give me an exception anymore... here is what the connection string looks like:
OleDbConnection conn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + dir + #";Extended Properties=""Text;HDR=Yes;FMT=Delimited""");
I assumed that if HDR=No and I removed the header (i.e. first line), then it should work... the only way to get rid of the exception if HDR=No is if I remove the constraints by calling ds.EnforceConstraints = false;
In any case, now I'm no longer getting the exception. If anybody can explain why this is happening, then please do tell.
Related
I haven been trying to figure this out all day and just couldn't find my mistake. I have an ms-access database and a c# program that is supposed to update the data in the db. Unfortunately I keep getting System.Data.OleDb.OleDbException: Syntaxerror in update statement.
I am using OleDbConnectionand OleDbDataAdapter and everything (except for one column) was working perfectly. I couldn't figure out why I kept getting the above mentioned error for that column (it was formatted just like the others -> text), so I deleted the column and created it again. Now I get the error for every single alteration I try to make.
public void changeFields(string id, string name, string famName, string department) {
DataSet myDS = new DataSet();
using (OleDbConnection conn = new OleDbConnection(connectionString)) {
conn.Open();
OleDbDataAdapter adapter = new OleDbDataAdapter();
adapter.SelectCommand = new OleDbCommand("SELECT * FROM Table WHERE ID = '"+ id +"', conn);
OleDbCommandBuilder builder = new OleDbCommandBuilder(adapter);
myDS.Tables["Table"].Rows[0]["Name"] = name;
myDS.Tables["Table"].Rows[0]["FamilyName"] = famName;
myDS.Tables["Table"].Rows[0]["Department"] = department;
adapter.Update(myDS, "Table");
conn.Close();
}
}
The names of the columns are correct and if I
Console.WriteLine(myDS.Tables["Table"].Rows[0]["Name"];
right before I try to update.
I actually get the correctly changed version. The error always occurs for line adapter.Update();
Is it possible that access rearranges or messes some index up when I insert a column into the database?
Help would be very much appreciated
I figured it out! The problem was that I had a reserved keyword as a column name. Now it all works perfectly fine
I am trying to insert new data into an old .dbf database created with foxpro.
The database has a lot of columns and I dont need to fill every single one.
The connection itself works. But now im getting the exception "Field XY does not allow null values" for every single one I'm not adding in my insert statement. But the database is configured to allow null values.
I am using the following code:
OleDbConnection dbfcon = new OleDbConnection("Provider=VFPOLEDB.1;" +
"Data Source=" + Application.StartupPath + "\\Daten;");
dbfcon.Open();
String query = "INSERT INTO TB_KUVG (KDNR, Kuvg_id) " +
"VALUES(?,?)";
OleDbCommand cmd = new OleDbCommand(query, dbfcon);
cmd.Parameters.AddWithValue("#KDNR", 1);
cmd.Parameters.AddWithValue("#Kuvg_id", 1);
cmd.ExecuteNonQuery();
dbfcon.Close();
So what am I doing wrong?
Is it better to use another way to write into a .dbf from c#?
You are almost doing it right. Note that parameter names are not important and would be located positionally (ie: #KDNR is added first so that it correspond to first ? placeholder). What you are missing was, if the fields you don't pass doesn't accept NULL values then you should notify the connection that instead you want "empty" values for those fields ('' for string, / / for date, 0 for numeric and vice versa). To notify the driver, you execute 'SET NULL OFF' on the same connection.
While adding it, I revised your existing code a bit:
string dataFolder = Path.Combine(Application.StartupPath, "Daten");
String query = #"INSERT INTO TB_KUVG
(KDNR, Kuvg_id)
VALUES
(?,?)";
using (OleDbConnection dbfcon = new OleDbConnection("Provider=VFPOLEDB;Data Source=" + dataFolder))
{
OleDbCommand cmd = new OleDbCommand(query, dbfcon);
cmd.Parameters.AddWithValue("#KDNR", 1);
cmd.Parameters.AddWithValue("#Kuvg_id", 1);
dbfcon.Open();
new OleDbCommand("set null off",dbfcon).ExecuteNonQuery();
cmd.ExecuteNonQuery();
dbfcon.Close();
}
PS: Application.StartupPath might not be a good idea as it may be under "Program Files" which is read only.
PS2: It would be better if you added "VFP" tag there, instead of "DBF".
I'm trying to query a CSV file. It works when I do a simple select, but as soon as I try to add a where clause, I run into No value given for one or more required parameters.
Obviously, it sounds like it's not getting the supplied parameter, but I've tried to pass it in a number of ways. See below for some code samples
DateTime lastRunDate = Convert.ToDateTime(ConfigurationManager.AppSettings["LastRunDate"]);
OleDbConnection conn = new OleDbConnection(
"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + base.applicationRoot + ";" +
"Extended Properties=\"text;HDR=Yes;FMT=CSVDelimited\"");
// This works just fine
//OleDbDataAdapter adapter = new OleDbDataAdapter(String.Format("select * from {0}",
// This gives the error
OleDbDataAdapter adapter = new OleDbDataAdapter(String.Format("select top 100 * from [{0}] where {0}.sale_date = #sDate", base.csvFileName), conn);
//adapter.SelectCommand.Parameters.Add("#sDate", OleDbType.DBDate).Value = lastRunDate;
adapter.SelectCommand.Parameters.AddWithValue("#sDate", lastRunDate);
// This also gives the same error as above
//OleDbDataAdapter adapter = new OleDbDataAdapter(String.Format("select top 100 * from {0} where sale_date = '{1}'", base.csvFileName, lastRunDate), conn);
base.csvFileName, lastRunDate.ToShortDateString()), conn);
DataTable dt = new DataTable();
adapter.Fill(dt);
I don't know anything about C#, and I'm still decently new to SQL, but perhaps it's the SELECT TOP part of your query. I know that SELECT TOP isn't really accepted on all db systems, and that it's included in both of your queries that are giving you problems. Have you tried removing that and using LIMIT instead?
"select top 100 * from [{0}] where {0}.sale_date = #sDate"
to
"select * from [{0}] where {0}.sale_date = #sDate LIMIT 100"
I would have added this as a comment as it's not a concrete answer, but I have not the required rep yet.:(
Remove this line. You have added parameter twice.
adapter.SelectCommand.Parameters.AddWithValue("#sDate", lastRunDate);
and make sure the value is present in lastRunDate variable. it should not be null.
EDITED:
Remove table name from the where condtion, Use like this
select top 100 * from [{0}] where sale_date=#sDate
Column Names in the Excel file and in the Query are not Same.
Either column name is missing.
Column Name not existing in the Excel File.
I found he issue with this. The query simply didn't understand the column names.
I thought that setting HDR=Yes meant that the oledb would read the first row headers, hence know them. But it wasn't until I added a schema.ini file that I managed to query in this way.
Here's some more about schema.ini files
I am using OLE related API's for reading data to DataSet and later to view in DataGridView. The source is an excel file. If the first row or column is empty than that row or column is getting skipped. I wish to read the data irrespective of empty/has data. Other rows/columns even if its empty its working fine, but only starting is missing.
I read about ColumnHeader and tried changing connectionString (HDR=NO) and others but nothing worked out.
Any additional things need to be specified while calling the API? One to one mapping with the excel column and the DataGridView column is missing because of skipping this initial blank rows/columns.
Anything needs to be added/modified to this OleDbDataAdapter parameter, which actually reads the data and fills to dataset:
OleDbDataAdapter cmd = new System.Data.OleDb.OleDbDataAdapter(
"select * from [" + worksheetName + "$]", con);
Yes, I am setting some required information and calling a method to read data from each sheet. I am not doing any action on the rows/columns except reading them.
OleDbConnection con = new System.Data.OleDb.OleDbConnection(connectionString);
System.Data.DataTable getWorksheetData(OleDbConnection con, OleDbDataAdapter cmd)
{
con.Open();
System.Data.DataSet excelDataSet = new DataSet();
cmd.Fill(excelDataSet);
con.Close();
return excelDataSet.Tables[0];
}
My connection string is:
string connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" +
fileName + "; Extended Properties=\"Excel 12.0 Xml;HDR=NO;Mode=Read;ReadOnly=True;\"";
As part of a project I'm working on in C# I need to read in a .dbf file. The first thing I want to do is to get the schema table from the file. I have code that works as long as the filename (without the extension) is not longer than 8 characters.
For example, let's say I have a file named MyLongFilename.dbf. The following code does not work; it throws the following exception: “The Microsoft Jet database engine could not find the object 'MyLongFilename'. Make sure the object exists and that you spell its name and the path name correctly.”
string cxn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyLongFilename;Extended Properties=dBASE 5.0";
OleDbConnection connection = new OleDbConnection(cxn);
To get past this exception, the next step is to use a name the OldDbConnection likes ('MyLongF~1' instead of 'MyLongFilename'), which leads to this:
string cxn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyLongF~1;Extended Properties=dBASE 5.0";
OleDbConnection connection = new OleDbConnection(cxn);
This does successfully return an OleDbConnection. Now to get the schema table I try the following:
connection.Open();
DataTable schemaTable = connection.GetOleDbSchemaTable(OleDbSchemaGuid.Columns,
new object[] { null, null, fileNameNoExt, null });
This returns a DataTable with no rows. If I rename the filename to 8 or less characters then this code works and I get back a row for each field in the database.
With the long filename, I know the returned connection is valid because I can use it to fill a DataSet like so:
string selectQuery = "SELECT * FROM [MyLongF~1#DBF];";
OleDbCommand command = new OleDbCommand(selectQuery, connection);
connection.Open();
OleDbDataAdapter dataAdapter = new OleDbDataAdapter();
dataAdapter.SelectCommand = command;
DataSet dataSet = new DataSet();
dataAdapter.Fill(dataSet);
This gives me back a DataSet containing a DataTable with all of the data from the dbf file.
So the question is how can I get just the schema table for the long named dbf file? Of course I can work around the issue by renaming/copying the file, but that’s a hack I don’t want to have to make. Nor do I want to fill the DataSet with the top 1 record and deduce the schema from columns.
According to MSDN, the folder represents the database and the files represent tables. You should be using the directory path not including the filename in the connection string then, and the name of the table as part of the restrictions to GetOleDbSchemaTable.
Well, i think the connection should be
string cxn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=C:\;Extended Properties=dBASE 5.0";
OleDbConnection connection = new OleDbConnection(cxn);
and the other is, maybe you should try with other provider, I boosted a lot along ago when I used like this:
string cxn = "PROVIDER=VFPOLEDB.1;Data Source=C:\;Extended Properties=dBASE 5.0";
But you should have VFP 7 installed
or install Microsoft OLE DB Provider for Visual FoxPro 9.0 from here
const string connectionString = #"Provider = vfpoledb; Data Source = {0}; Collating Sequence = general;";
OleDbConnection conn = new OleDbConnection(string.Format(connectionString, dirName));
conn.Open();
OleDbCommand cmd = new OleDbCommand(string.Format("select * from {0}", fileName), conn);
Is fileNameNoExt holding the short filename version? Also, MyLongF~1 is 9 characters, not 8.
If you have a single (and possibly small) dbf file you can solve the problem copying the dbf file elsewhere and open the copy instead of the original file.
I believe that the DataSource should represent the directory that contains the .DBF files. Each .DBF file corresponds to a table in that directory.
My guess is c:\MyLongF~1 is a short name for a directory that contains a filename corresponding to MyLongF~1#DBF
Can you verify whether or not this is the case?