How do I convert an exported IXF file (using db2 export) to a human-readable format, like CSV or XML? I am comfortable with doing it in Python or .NET C#.
The PC/IXF format is fairly complex, and is practically unknown to programs outside of DB2. Writing your own PC/IXF parser just to convert an IXF file directly to some other format might take a while. A faster alternative is to issue an IMPORT command on the DB2 server and specify CREATE INTO instead of INSERT INTO , which will generate a brand new table that can accommodate the contents of the file being imported. This will allow you to run an EXPORT command on the new table to dump the rows to a delimited format.
In case you still want write your own PC/IXF parser, you can take a look at this project, that converts IXF file to JSON
IXF is an old and also well documented file format. It is possible to read and process it, I've done this couple of years ago. So don't let you be discouraged. It's hard, but not too hard for developers.
Today you can find also solutions on GitHub, e.g. ixfcvt
Related
For reasons best known to a supplier I am being provided a number of .CSV files in both ANSI and UTF formats and this is confusing my flat file load process. Basically, I cannot rely on the format being the same each time, although the structure is consistent.
So my questions are:
Does the fact that my flat file process has issues with (UTF and ANSI) mean that I have not set up the flat file connection properly?
I believe I have identified two possible solutions, but which of these solutions would be best?
A split by type (ANSI vs UTF) and if so how?
Convert all the .csv files to "excel" as part of a VB/C# script task? For "excel" read any other common format.
Thanks for your help.
Flat file connection manager change code page to 65001 (utf -8)
sample com
I have to create a fix length record file using C#. I read the records from database and then after some business logic I would like to write some of the fields into a text file.
I researched this and some recommend using an XML to define the file, while others format the string and then write that into a file.
Is there a proper way to do this? I would like to keep this object oriented.
Thanks for any hints.
take a look at http://www.filehelpers.com/ to export the data to a fixed file format. you may also want to look at http://hibernatingrhinos.com/open-source/rhino-etl to create a process that runs the export for you. rhino.etl includes operations for FileHelpers.
You can use Jet to parse fixed length records in a text file. This is a decent overview that should be able to get you started:
http://msmvps.com/blogs/deborahk/archive/2009/08/25/reading-fixed-length-files.aspx
In the past when i've done this, I pull the data out of SQL as a Char(Length) so that the padding is handled by SQL. This makes the C# pretty easy to put together. You can do this by casting or converting the data when you select it, or by creating a table that is the exact format of your file and inserting the data in there before pulling it out into C#.
I have a task at hand, read a database that was created with HTML-OS, i assume the format is DB4 or DB5. the task is to open and parse this database in a c# application...it can be ASP.NET or WinForms...bottom line is i need to extract this indexed data. below is a small sample of what it looks like when i open the file with notepad:
dbtype 3.046 = 0 T ¨j
I have some ideas on parsing with possibly using RegEx but i would like to see some ideas or a real way of reading this file would be even better!
Thank you in advance!
It looks like the database format is partially defined on their site:
The HTML/OS Database (PDF)
If possible I'd look at using their HTML/OS language to export it, rather than trying to reverse-engineer the database format.
You could make an htmlos page that would export the data from the database and store this in a csv for example. Then pick up the file from the other machine, after which you could execute something else to remove the csv file.
To answer devHead's question.. yes, HTML/OS accepts http posts.
(Just FYI: If you have any questions, feel free to ask.. I program in html/os.)
Currently our application exporting large data to office 2003 xml spreadsheet format in server.
The user can download the xml file.The file can be easily opened in office 03 and 07 correctly.
I want to know whether it is possible to create xls and xlsx format from this xml file in server and serve them to user?
[The server doesn't have office and neither will be.So interop is useless in this case.]
====== EDIT=====
Can't use third party solutions... :(
With your XML file, you could easily use XSLT to transform your XML into CSV. However, if you really need to export in native Excel spreadsheet types, it's probably a good idea to use a component.
Something we've used in the past, that I'd recommend, is Aspose.Cells – we've used this to generate Excel spreadsheets where plain CSV output wouldn't suffice (are you sure it won't suffice in your case - XSLT will make short work of that).
Whilst it's not free, it saves having to install Office on the server (and avoid using Office Automation etc.) and has very good support.
Edited following comments:
If you can't use third party components, if you can't install Office on the server, and you can't use interop, then – unless you want to get comfortable and read the Excel Binary File Format (.xls) Structure Specification as provided by Microsoft – the answer is a very short one: no.
If I understand your question corretly, you are not just serving 'pure' XML, but SpreadsheetML that is interpreted by Excel just as an ordinary Excel-file in binary format. So your users already have a very nice solution. How do are you creating those files? I use an XSLT to transform my XML-data to SpreadsheetML that is just a specific XML-format. In that way I can do anything that is possible in Excel (formattings, formulas etc.). It yould be much more than just transforming XML to CSV.
Why do you want to change this? I can see that the resulting XML-Excel file usually is huge, is that the reason?
Now, since Excel 2007 format (xlsx) is basically a zip-archive of some XML-files describing the structure and the data of the file -- just rename the file extension name of any xlsx to zip and inspect after extracting -- you might be able to adapt your existing solution, i.e. instead of creating one SpreadsheetML-(XML)File create several and pack/zip them one the server. I assume your normal server installation should have the necessary tools.
BUT - this solution is difficult to maintain (as you might know wiht you current solution -- if I am at all right) and changes in the XSLT to add new functionality requires quite a bit of knowledge. So again I as well can recommend Aspose.Cells, even some knowledgeable users might be able to change simple formats.
HTH
Andreas
If you don't have office 2003 *or later) DLLs referenced to the project, I'm afraid you can't read that Office XML data in your program directly, unless of course you can manually parse the whole XML by studying its format.
You can use Open XML SDK to create or read xlsx files.
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=5124
I generate files, lets call them .dwrf files, which contain a significant amount of data. Currently we export those to .CSV and the resulting files are large (2GB+). I would like to cut out the export process and make the contents of a .dwrf file queryable directly from Excel or other applications.
What I would like to do is write a utility/service - lets call it dwrfMiner - to extract data from the file and pass it on as a datasource and link dwrfMiner to .dwrf files in some way so that Excel recognises it as an external data source.
Any ideas?
While writing an ODBC driver for this is probably overkill, if the format of the files you are working with is known in advance and isn't too hard to translate (it sounds like not considering you are already creating CSVs) then using an ODBC DSN sounds like your best bet.
There are a nice selection of ODBC drivers already built in to Windows (.txt, .csv, .mdb, .xl*, .dbf, Paradox .db, etc etc) and you can obtain other drivers from the web for a lot of common formats.
If the size of the existing format you're exporting to is too onerous (CSV) then the logical point to start is a transformation of your data to something more space-conscious that has ODBC support.
Failing that, your last option is the overkill option (Writing an ODBC driver).
Excel can query external data souces, but beware that Excel (all versions) have hard-limits on the number of rows they can display, per work-book. I think in Excel 2003 the limit is ~65k. It's higher in other versions.
See my question: reporting tool/viewer for large datasets (and I had much less than > 2GB).
I used PHP FlatFile DB to query flat-files in the past
I'd get out gcc and write yourself a full ODBC driver for it. Then you can sit back and use SQL.
You know, if you're bored. ;)
use odbc driver with multithreading