Develop Reference

Generating report or spreadsheet from simple XML listing of data

I have XML objects that hold form data from a web application. These are as simple as can be for an xml format, a demonstrative example would be:
<data>
<patientFirstName>Bob</patientFirstName>
<patientLastName>Bobson</patientLastName>
</data>
and unfortunately due to humans beyond my control, roughly 150 other potential fields for the current "document".
The primary goal is to transfer this data from XML to a more "human-friendly" format such as a spreadsheet or PDF, etc..
I have tried using a DataSet as the source for a DataGrid and producing an html table. This works for a browser, and technically loads in Excel - but Excel gives a file type error, as the file is not in Excel format, it is literally an HTML table.
Is there any way to use the .net framework, or other MS objects whether managed or (preferably) unmanaged to take XML or a DataSet and turn it into a useable file format by typical office applications?
Is there a way to use SSRS without SQL queries or files? My big issue is this is a small Information System that does many things dynamically. I have seen examples of creating Excel spreadsheets using a file on the disk...but it is 2019 - I will allow my code to create or read files, the concept of a file is quite obsolete and a major source of security dilemmas.
In a perfect world I would want to create a decent looking report form template and simply bind certain nodes to their respective node in the XML. This would seem like something that many people would want to do, and as if it would be easy to implement, but search as much as I may and I find 99 terrible ways to pretend to make an Excel file and not 1 decent one. SSRS would seem like a lovely option if I could request to use a premade template and pump XML into it or a DataSet from a middleware server - though my issue is that it seems to be dumbed-down to the point it is designed to do ALL the work in some not-very-flexible manner.
If all else fails, might there be some MS functionality for pagination that I can use to create my own mini-report generator? I feel I may be forced to..
Please understand I am not looking to hand-parse markup in C# - in unmanaged C++ I would consider it, but C# becomes very slow when you start doing things "by hand" as opposed to using objects and their methods.
.net 4.5+

Need to automate crunching of excel data?

I have processes running on Windows XP/7. They generate weekly .csv data files. I have a bunch of excel formulas that crunch the numbers for each .csv file produced for the week separately and then when adding the weekly data to the one big spreadsheet containing all the data put together.
The number of rows varies each week and for each process. So I can't hardcode that number in my dozens of formulas. So right now I go through this stupid process of manually entering the formulas each week into the .csv files.
There's got to be a way of automating this. Just now I quickly looked into doing this through C# or VB code. Could somebody recommend the best way to do this. Is C# or VB the right way to go? If so, any hints on how to put it all together - what's the model to use? For example, would it look something like this:
C# module reads in .csv data file
C# module creates an Excel spreadsheet and populates it with the .csv data
C# module runs my formulas on the all the rows.
Is that how one would approach it? Is there a better way for somebody who has very limited knowledge of C# or VB? I know Java and C++.
Any advice would be highly appreciated.
Thanks

From your explanations in comments, it appears that having a series of template Excel sheets would greatly facilitate the task.
So, for each process that generates data, you say the formulas are always the same, meaning that the columns are always the same (am I right?).
So, even if you don't know how many rows of data, you can still either create a template where only the first row is filled with formulas, and then you simply copy that row over and over, filling it with data as needed, or, you could fill a relatively "comfortable" number of rows with those same formulas, and fill in the data.
There are tons of atricles on how to Interop with Excel, so it's beyond my intent to provide you with specific code, but the idea is good.
If I can allow myself, I have worked in the past with a very interesting tool call Flexcel Studio for .NET, and I have found it to be of great help when it came to generating Excel sheets based on such templates.
Cheers

As others have suggested, I would recommend performing the calculations outside of excel if possible. There are plenty of stats libraries out there that are friendlier to work with than going through the hassle of moving data into excel, applying formulas to cell ranges, and so on.
If you really want to go the excel route, you can either use open-source libraries such as EPPLUS (.NET) or POI (Java) to work with .XLSX files directly. Some libraries do not support function evaluation so you will need to consider this when deciding on a library to use.
If you go with COM interop, you should read about about the following: Considerations for server-side Automation of Office.
As for the C# or VB (if not java with POI), I would go with C#. C# syntax is similar to java.

There might be a really simple solution to this problem.
Add 1 piece of auxiliary data to the .csv file either programmatically when running my process or when creating the .xlsx file (with all the formulas) from the .csv file. The auxiliary piece of data is the row count which will be in some known location.
Then modify all my formulas to use the INDIRECT function to specify the range using the cell
with the auxiliary piece of data.
I think that might work.

Fix length record and creating a file

I have to create a fix length record file using C#. I read the records from database and then after some business logic I would like to write some of the fields into a text file.
I researched this and some recommend using an XML to define the file, while others format the string and then write that into a file.
Is there a proper way to do this? I would like to keep this object oriented.
Thanks for any hints.

take a look at http://www.filehelpers.com/ to export the data to a fixed file format. you may also want to look at http://hibernatingrhinos.com/open-source/rhino-etl to create a process that runs the export for you. rhino.etl includes operations for FileHelpers.

You can use Jet to parse fixed length records in a text file. This is a decent overview that should be able to get you started:
http://msmvps.com/blogs/deborahk/archive/2009/08/25/reading-fixed-length-files.aspx

In the past when i've done this, I pull the data out of SQL as a Char(Length) so that the padding is handled by SQL. This makes the C# pretty easy to put together. You can do this by casting or converting the data when you select it, or by creating a table that is the exact format of your file and inserting the data in there before pulling it out into C#.

Open a HTML-OS Database (DB4/DB5) and parse/read in c# application

I have a task at hand, read a database that was created with HTML-OS, i assume the format is DB4 or DB5. the task is to open and parse this database in a c# application...it can be ASP.NET or WinForms...bottom line is i need to extract this indexed data. below is a small sample of what it looks like when i open the file with notepad:
dbtype 3.046 = 0 T ¨j
I have some ideas on parsing with possibly using RegEx but i would like to see some ideas or a real way of reading this file would be even better!
Thank you in advance!

It looks like the database format is partially defined on their site:
The HTML/OS Database (PDF)
If possible I'd look at using their HTML/OS language to export it, rather than trying to reverse-engineer the database format.

You could make an htmlos page that would export the data from the database and store this in a csv for example. Then pick up the file from the other machine, after which you could execute something else to remove the csv file.
To answer devHead's question.. yes, HTML/OS accepts http posts.
(Just FYI: If you have any questions, feel free to ask.. I program in html/os.)

Reading large text files to datagridview a filtering

I need to read line by line from text file (log files from server) and they are big (about 150-200MB). I am using StreamReader and its great for "little" files like 12MB but not for so big. After sometime it is loaded and it shows in my DataGridView but its huge in memory. I am using bindingSource.Filter on this DataGridView (like textbox and when user write letter it is filtering one column a comparing strings, not showing rows without letters in textbox and so) and with big files its useless too. So I want to ask you what is best solution for me.
I was looking and find some solutions but I need help with decided whats best for me and with implementing (or if there is something else):
Load data in background and showing them in realtime. I am not really sure how to do that and I don´t know what to do with filtering in this solution.
Maybe upgrade somehow streamreader? Or write own method for reading lines from file with binary readers?
I found something about Memory-Mapped in c# 4.0 but i can´t use 4.0. Could this help feature help?
Thanks for help
Okay, so I am implementing Paging and I read 5k lines of text file than after clicking button next lines and so. I am using BaseStream.Position for saving a starting reading but I would like to use some other function which save number of lines and mainly I want use method for starting reading from exact line but I can´t find nothing for StreamReader. Is there something like that?

Load data in background and showing them in realtime. I am not really sure how to do that and I don't know what to do with filtering in this solution.
This is no help. It will still consume much memory in the background thread.
Maybe upgrade somehow streamreader? Or write own method for reading lines from file with binary readers?
Still no help, once you read the whole file into memory it will, well, consume memory.
I think you get the point. Don't load the whole file into memory. Load only chunks of it. Use paging. You cannot show 200MB worth of data on a single screen anyways, so only load the portion you need to show on the screen. So basically you need to implement the following function:
public IEnumerable<string> ReadFile(int page, int linesPerPage, out totalLines)
{
...
}
The Skip and Take extension methods could be helpful here.