I'm attempting to select the columns I want from a FoxPro DBF table and insert them into a MySQL table, preferably (if possible) creating the SQL Table on the fly from the columns I've chosen.
I've had a look and I unserstand this is probably going to be built around the SQL Statement "SELECT INTO FROM" but I'm unsure how I would do this, especially programatically within C# and especially scenes as I'm going from DBF to MySQL, not MySQL to MySQL.
Can someone advise on how I would do this within C#? I already have an OleDB Connection setup as such;
public void runDbfCmd()
{
string constr = ConfigurationManager.ConnectionStrings["dbfString"].ConnectionString;
using (OleDbConnection dbfCon = new OleDbConnection(constr))
{
try
{
dbfCon.Open();
OleDbCommand dbfCmd = new OleDbCommand("SELECT INTO STATEMENT HERE");
dbfCmd.ExecuteNonQuery();
}
catch (OleDbException ex)
{
throw ex;
}
finally
{
dbfCon.Close();
}
}
}
As you can see I'm stuck on how to build the command and whether or not this will work how I would like it to.
If you want to do it on the FoxPro side by sending SQL statements there, the FoxPro statement Select ... From ... Into ... as you proposed would probably not do what you want because it would create a FoxPro "Into" destination, i.e. a DBF table or a "cursor" or an array.
Whereas on the FoxPro side, SQL Insert Into ... Select ... From ... would be able to do what you want if the "into" destination is a Visual FoxPro "CursorAdapter" object, or a "Remote View" (residing in a FoxPro "database" DBC), which both could be connected to a MySQL database via ODBC, and the former also via OleDB. If you do not have either of them already prepared on the FoxPro side, you would need a VFP development environment to create them because not all FoxPro commands and functions are supported by its OleDB driver, see also "Supported Visual FoxPro Commands and Functions in OLE DB Provider" https://msdn.microsoft.com/en-us/library/80x51c04%28v=vs.80%29.aspx
Assumed you do not have VFP, perhaps it would be easier to do most things on the C# side?, like sending "Select ... From ..." statements to the FoxPro DB, retrieving and intermediately storing the result on the C# side, and then send "Insert ... Into ..." statements to the MySQL DB, and also "Create Table ..." statements if you will
What you are looking to do can not just be done from C# from a VFP database and directly into SQL in one statement as you are attempting. Reason being. You can't have one connection pointing to two different database types... VFP and MySQL respectively.
What you would need to do is...
Open Connection to VFP,
select * from YourTable and fill into a C# data table...
Look into OleDbDataAdapter.Fill()
Once the data has been pulled from VFP locally, you can then open a connection to your MySQL Database, create the table, and then for each row in the localized VFP table, insert into the MySQL.
Alternatively, once local in C#, you could possibly dump to a standard format that MySQL can do a bulk insert into.
But again, you can't go directly from one to the other.
Related
I have thousands of records from a SQL Server 2014 stored procedure result set, and I insert them one by one into a SQLite DB table through C# code, which takes around 4-5 minutes. I need to reduce this time.
I am looking for something like:
insert into 'sqlite_table'
select *
from 'sql_server_table'
Any answer with C# code or anything direct from SQL Server script can be helpful
On your C# Code. Use SqlTransaction to fasten the insertion of records from one table to another.
I Have done same thing in Sql Server to MySql. For that first I stopped the AutoCommit (Set AutoCommit=False). After the query run the commit command. It gave me good performance. You can try the same if it works for you.
To explain, I have a database db_Name. Over the course of development of my program, the parameters of this database has changed through the use of MySQL Workbench. Unfortunately the hard-coded construction of the database in my program has not been kept up to date.
Is there a way that I could return the code for the construction of my database in the latest version?
To explain: I'm looking for this code:
Table_Name_1(Tbl_Id CHAR(25) NOT NULL AUTO_INCREMENT, Field_1 CHAR(25) UNIQUE, Field_2 INT, PRIMARY KEY(Tbl_Id));
Table_Name_2(..........
etc.
If you need SQL for database structure - you can backup your database to *.sql file without table data.
You can do that in MySQL Workbench.
Step-by-step guide:
How to take MySQL database backup using MySQL Workbench?
You can also use SHOW CREATE TABLE TableName sql statement to return database structure as data view.
I have a Windows Service application that receives a stream of data with the following format
IDX|20120512|075659|00000002|3|AALI |Astra Agro Lestari Tbk. |0|ORDI_PREOPEN|12 |00000001550.00|00000001291.67|00001574745000|00001574745000|00500|XDS1BXO1| |00001574745000|ݤ
IDX|20120512|075659|00000022|3|ALMI |Alumindo Light Metal Industry Tbk. |0|ORDI |33 |00000001300.00|00000001300.00|00000308000000|00000308000000|00500|--U3---2| |00000308000000|õÄ
This data comes in millions of rows and in sequence 00000002....00198562 and I have to parse and insert them according to the sequence into a database table.
My question is, what is the best way (the most effective) to insert these data into my database? I have tried to use a simple method as to open a SqlConnection object then generate a string of SQL insert script and then execute the script using SqlCommand object, however this method is taking too long.
I read that I can use Sql BULK INSERT but it has to read from a textfile, is it possible for this scenario to use BULK INSERT? (I have never used it before).
Thank you
update: I'm aware of SqlBulkCopy but it requires me to have DataTable first, is this good for performance? If possible I want to insert directly from my data source to SQL Server without having to use in memory DataTable.
If you are writing this in C# you might want to look at the SqlBulkCopy class.
Lets you efficiently bulk load a SQL Server table with data from another source.
First, download free LumenWorks.Framework.IO.Csv library.
Second, use the code like this
StreamReader sr = new TextReader(yourStream);
var sbc = new SqlBulkCopy(connectionString);
sbc.WriteToServer(new LumenWorks.Framework.IO.Csv.CsvReader(sr));
Yeah, it is really that easy.
You can use SSIS "Sql Server Integration Service" for converting data from source data flow to destination data flow.
The source can be a text file and destination can be a SQL Server table. Your conversion executes in bulk insert mode.
I'm using .NET 2.0 and/or 3.5. Weird thing is, everytime I add a query, whether via TableAdapter or a plain query in a Dataset, using a MySQL stored procedure (whether be a select, update, insert, delete), the wizard doesn't finish (the dialog suddenly disappears, I'm back to designer mode and the query isn't added to the tableadapter or dataset form). Is there a special formatting required for MySQL stored procedures, or a workaround for MySQL stored procedures to work?
I'm using
MySQL 5.1.33
portable XAMPP 1.7.1
PHP 5.2.9
Apache 2.2.11
phpMyAdmin 3.1.3.1
I was having this same thing happen. The fix for me was the following: One of my parameters for the stored procedure was 'filter'. I changed it to 'p_filter' and the issue went away. The issue that I am still dealing with is on the last screen of the filter I get:
you have an error in your sql syntax; check the manual that corresponds to your mysql server version for the right syntax to use near 'NULL' at line 1
I brought in a stored procedure that doesn't accept a parameter and I have no issues with creating the TableAdapter using the dataset wizard.
I have a CSV file at the client side, and I want to develop a C# application to bulk insert the data into a table of a database to minimal log output. I am confused about if I use ADO.NET at the client side to call stored procedures in the database server. What kind of code needs to develop at the client side and what kind of code needs to be implemented at the server side in the form of stored procedures?
But I did not find any samples from Google. What are some ready to use samples? :-)
EDIT: Some more information:
I have a lot of data at the client side and I want to import to the database, but I do not want the overhead of all the many transaction logs. For security reasons, I want to develop a stored procedure at the server side and call from client side (ADO.NET). I want to know to achieve such goal. What kind of T-SQL needs to be developed in stored procedures at the server side and how to call/fill data efficiently at the client side?
If anything is still unclear, please feel free to let me know.
You can hook CsvReader to SqlBulkCopy, which does the job very nicely... something like (untested):
using (CsvReader reader = new CsvReader(path))
using (SqlBulkCopy bcp = new SqlBulkCopy(CONNECTION_STRING))
{
bcp.DestinationTableName = "SomeTable";
bcp.WriteToServer(reader);
}
edit you would typically do the bulk-insert into a staging table, and then use a regular stored procedure to move the data into the real table.
Are you using SQL Server 2008? And are you able to execute dynamic SQL (not that I'm advocating it)?
If so, you could construct an insert statement that makes use of "Row contructors". Essentially an insert statement will now accept an array of arguments for each row, like so:
INSERT INTO TableA (Col1, Col2)
VALUES ('A', 'B'), ('C', 'D')
There's more about it in the blog post "SQL Server 2008 – Insert Multiple Records Using One Insert Statement – Use of Row Constructor".
I hope this helps.