I'm using Visual c# express 2008.
I have a huge text file that has data similar to this: "text/text/text/text"
I'm using a delimiter to separate the data.
Now I want to transfer this data to an .sdf file that will be displayed in table format through the windowsformapplication using dataGridView.
I already created an sdf with a table (and columns).
I know how to access the .sdf and display through the dataGridView.
My problem is I have no idea how to transfer the data from the txt to the sdf.
I don't want to do it manually because the txt file contains around 20,000 worth of lines.
Resources: I've recycled code from
http://dotnetperls.com/datagridview-tutorial
Thanks to anyone who can help. :3 Ree
From memory (not tested) SqlBulkCopy works with SDF, so then all you need is an IDataReader. If you search for "fast CsvReader" you should find one on codeproject; this works with any delimiter and should work perfectly with SqlBulkCopy (I've used it this way multiple times, although not with sdf).
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopy.aspx
http://www.codeproject.com/KB/database/CsvReader.aspx
Also, you could try my new Csv2sqlce utility: http://csv2sqlce.codeplex.com
Related
my ado application requires to read data from one xlsx file(Approx 10-20MB min) and then process the data row by row and compare with another xlsx file( approx size 250 MB min) containing over 1000000 rows with 63 columns (work like database). when i try to read the database file (250 MB) and run the oledb data queries on it, it's working so strange, its only give the first matched data from the database file. but if i opened that xlsx database file in office excel and then run my application its return all matched data from the database file without changing any code.
i already checked my dataquery its working fine in server Explorer and returns the complete result.
this process also very time consuming , i already tried openxml sax mathod to resolve the performance issue as well, but its not worked too. even it takes more time compare to oledb to read the excel.
i also have two another issues in my application,
sometime oledb returns an exception 'System resource exceeded' and
some time return 'internal ole automation error' .
i also tried google to resolve these issue but i didn't find any solution for my problem.
is there any solution to resolve these issues. please help me. any suggestion appreciated but please remember one thing i can't change any xlsx table format because i don't have rights, these xlsx files automatically generated by another tools by our sourcing partners.
thanks.
I am wanting to add the ability to import massive CSV files into a table in the database. I've built an SSIS package that does this, but I just wanted to make sure that this was the correct way of doing it.
The text files are millions of rows with 50 columns. They don't open in notepad or notepad++ a lot of times. The SSIS package handles them with no problem and gets everything imported. Is SSIS the right way to deal with this? I just need to pass the file location parameter to the job and execute it right? Is there an easier way that I am overlooking?
The text files are millions of rows with 50 columns
Small. Why an SSIS package?
They don't open in notepad or notepad++
because both OPEN them - there is no need to open them and load them all into memory, a proper application can read them line by line.
Is SSIS the right way to deal with this?
No. SImply because the title is: Importing huge data from text file to sql using c# winforms
As Winforms can do that effectively - I am inserting around 100.000 rows into a database - in C# with quit eeasy coding (except some pages and a day to get sqlbulkcopy to work properly) and you say nothing about any transformations, SSIS just is another not needed technology AND makes things complicated (as in: more to install, or hav a packag on the server but then find a location for the file that the server can reach etc.
I am all for SSIS and if you have a larger SSIS infrastructure or do a lot of processing etc. and it makes architectural sense in a larger context - yes. But as the question stands, without additional reasons - absolutely not.
Heck, there is a good chance you can load the whole thing in one command into SQL server as SQL Server has some CSV processing capabilities:
http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
I am making a window application in asp.net using C#. I want to browse and import .csv file and save it in the database. Importing part I have done. Now, how am i supposed to save it in the database ? I am new to the language so please help
You can use ADO.NET, you will need to look up Data Adapters and the like.
If your Importing the Spreadsheet into a DataTable or DataSet you can use a Data Adapter to populate a table in the Database using an INSERT command.
Look up http://msdn.microsoft.com/en-us/library/system.data.common.dataadapter.aspx
Hope This Helps.
Once you have a file upload mechanism in place, (for example, using the FileUpload control), just iterate through the lines in the CSV file on the server, parse them into fields using an existing CSV reader (don't try to code your own unless your data is completely trivial, you won't handle all of the edge cases), and just execute a bunch of SQL INSERT statements against your database using your calling convention of choice.
I am using DataGridView and i am binding my Excel file to DataGridView.
I am able to read from the Excel file as well as insert to that file using DataGridView. But I am not able to delete and update that file.
It's giving an error that says:
Deleting data in a linked table is not supported by this ISAM.
Is there any other method to perform these operation in an Excel file using DataGridView. Or simply how can I delete and update a record in an Excel file using DataGridView in C#.
I am afraid that a normal delete (like from access for example) is not supported as mentioned here by microsoft, i think you should consider using MS Access as your data source since it is much easier to deal with or you have to work around a different approach like deleting the while file and recreating it each time for both deleting and updating (not recommended).
In the page I linked they said the best way is to empty the cells for the row u need to delete.
good Luck
I am trying to figure out an eazy way to load CSV file into SQLite DB using System.Data.SQLite
I saw commandline way to do that i.e .Import mydata.csv mytable
But I need to do this via C# code. Any Idea?
you can create OleDbConnection to CSV file (just google it, it is very easy) then load rows to DataSet, then put that dataset into Sqlite by SqliteConnection. Few lines of code.
You could use the file helpers library if you want also.And well use the ado.net functionality too.