Hey guys, I need to export data from database which contains Unicode(like Japanese characters).
While when i trying to export to csv files, it just becomes the garbage....
First, I need to read data from database and save it to an datatable
Second, I will make a query based on the datatable (Using Linq)
Last, Export the queried results to csv file
Is there any way to solve this problem?
Thanks a lot.
Internally .Net (C#) always uses unicode. The problems occur when you export the data. Then you need to specify an encoding.
If you use File.CreateText you immediatly create a UTF-8 encoded file.
Does the application you use for import, recognize the UTF-8 encoding, or does it need a specific encoding?
Try to set connection data collation to unicode_ci or more specidfied.
Related
I have a process in SSIS that outputs SQL table data to CSV format. However, I want the output CSV in CSV (MS-DOS). Is there a way I can convert the normal CSV file to CSV (MS-DOS) ? (Like C# code that would convert the extension/type) . I tried using the option available in visual studio in SSIS, and couldn't find the solution towards it. Your help is appreciated.
By default, the output format is in CSV(Comma delimited, highlighted blue). I want that to be converted to CSV(MS-DOS, highlighted yellow).
If this article is accurate, https://excelribbon.tips.net/T009508_Comma-Delimited_and_MS-DOS_CSV_Variations.html then getting an CSV (MS-DOS) output will be fairly straight-forward
if you have certain special characters in text fields; for example, an accented (foreign language) character. If you export as Windows CSV, those fields are encoded using the Windows-1252 code page. DOS encoding usually uses code page 437, which maps characters used in old pre-Windows PCs.
Then you need to define 2 Flat File Connection Managers. The first will use 1252 (ANSI - Latin I) as your code page and point to C:\ssisdata\input\File.csv. The second will use 437 (OEM - United States) and point to C:\ssisdata\input\DOSFile.csv (this way you create a new file instead of clobbering the existing.)
Your Data Flow then becomes a Flat File Source to Flat File Destination.
I am looking to import a csv file in a windows form, datagridvew1 and then parse/update some of the columns into datagridview2, and finally write the output as csv
I am struggling to convert the required fields from datagridview1 into datagridview2
Any suggestions welcome.
Thanks
There's a million ways to do this but my suggestion would be assuming you are writing your code for windows use the microsoft text driver to load the data directly from the text file into a dataset (roughly as shown in: CSV upload in .NET using ODBC including the answer's fix for the bug in the posters code). Then clone and dump out the data into the second dataset/csv file as so: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/dataset-datatable-dataview/copying-dataset-contents .
Lastly: I would suggest not blindly looping though the datarows and outputting row[i] + "," etc like a see a lot of posts on similar questions suggesting. The problem is escaping fields that contain commas themselves properly etc. Better to use a library, one I've used in the past and found nice is CsvParser you can get that via nuget. It's better to use libaries anyways because when the next complication comes up like utf-8 encoded text often you just have to pass in an additional parameter rather than learning and properly covering all the edge cases of how to do the encoding yourself.
I am trying to fetch data from database and export it to .csv file format, which is similar to ETL process. But I want to do this in C#.
SQL Query to fetch data from database.
Format the data to a specified file specification.
Convert it to .csv file.
I know that 1st step is easy to do, I am struggling to find a way for 2nd and 3rd step.
I can't comment on the file specification, as you haven't described it, but writing CSV files is pretty easy. Unlike XML etc the format is so simplistic you can write directly to a StreamWriter using WriteLine. All you need is to output a first line that contains the column names separated with commas, then for each row returned from your SQL Query write the column values in the same order separated by commas. The only real gotcha is escaping, e.g. dealing with commas, quotes, etc by surrounding each value with quotes and escaping any quotes in the value.
The example below does just that for a DataTable:
http://dotnetguts.blogspot.co.nz/2007/01/exporting-datatable-to-csv-file-format.html
I figured out a simple and efficient way to do this:
1.SQL Query to fetch data from database.
2. Format the data to a specified file specification.
For the first 2 steps, I have used the Dapper.NET, which
took care of the database part as well as formatting. Dapper helped to convert the SQL results to a LIST by which I fulfilled the file specifications.
3.Convert it to .csv file.
For converting the SQL results to CSV file, I have used FileHelpers
Library which is more simple than I expected.
Hope it will help someone.
I have to create a fix length record file using C#. I read the records from database and then after some business logic I would like to write some of the fields into a text file.
I researched this and some recommend using an XML to define the file, while others format the string and then write that into a file.
Is there a proper way to do this? I would like to keep this object oriented.
Thanks for any hints.
take a look at http://www.filehelpers.com/ to export the data to a fixed file format. you may also want to look at http://hibernatingrhinos.com/open-source/rhino-etl to create a process that runs the export for you. rhino.etl includes operations for FileHelpers.
You can use Jet to parse fixed length records in a text file. This is a decent overview that should be able to get you started:
http://msmvps.com/blogs/deborahk/archive/2009/08/25/reading-fixed-length-files.aspx
In the past when i've done this, I pull the data out of SQL as a Char(Length) so that the padding is handled by SQL. This makes the C# pretty easy to put together. You can do this by casting or converting the data when you select it, or by creating a table that is the exact format of your file and inserting the data in there before pulling it out into C#.
Can you please suggest how can we export the oracle clob data field to csv file. The table contain other regular data types too. And also if the clob data cannot fit in a csv what are the other options available?
Thanks, Naveen
You can read clob's just like strings. new System.Data.OracleClient.OracleConnection() and so on. When you have the data, you can write CSV's with a StreamWriter because they are just text files.
Storing CLOBs in a CSV file is possible if you use a serious CSV writer that supports:
Multi-line fields
Doubling of double quotes
Setting the encoding
Then you can read the exported file in any other application that has a serious CSV parser, such as Excel.
Just don't go for any CSV classes that have the word simple in their name. They certainly will be too simple.