I have a requirement to generate output report in Excel format and open the same on the screen when the processing is complete. But in this case, it should not save the report on the drive anywhere and only open on the screen.
I tried to use ADO using OLEDB but it always generates file before writing anything to it.
This is what I have tried so far.
using (OleDbConnection con = new OleDbConnection(connString))
{
try
{
con.Open();
}
catch (InvalidOperationException invalidEx)
{
//Exception handling
}
// Create table for excel structure
StringBuilder strSQL = new StringBuilder();
strSQL.Append("CREATE TABLE [" + tableName + "]([TITLE] text,[SURNAME] text,[STATUS] text)");
// Define file columns
StringBuilder strfield = new StringBuilder();
strfield.Append("[TITLE],[SURNAME],[STATUS]");
OleDbCommand cmd = new OleDbCommand(strSQL.ToString(), con);
cmd.ExecuteNonQuery(); // This creates the table
//Actual row for creating and insering row - logic not shown completely
cmd.CommandText = strSQL.Append(" insert into [" + tableName + "]( ")
.Append(strfield.ToString())
.Append(") values (").Append(strvalue).Append(")").ToString();
success = cmd.ExecuteNonQuery();
But this always creates the file first which I do not want.
Please advise if anyone has worked on the similar requirement. Thanks.
Ok, first off use ADO (a database access technology) to try and create a spreadsheet is bizarre, possibly doable, but definitely not easy.
Secondly you're saying create a spreadsheet and open it, without creating a file, this means that you'll also have to create ALL the functionality to open, parse, format and display spreadsheets (basically recreate Excel!)...as Excel cannot do this for you.
So I would question the "generate output report in Excel format" requirement, does this really mean "display in a grid"? Or is it "display in a grid that allows formatting, totalling?"
If it the Excel format really is a requirement, then the only thing I can suggest is you will have to create a temporary Excel file, then delete it after you've displayed it.
I would look at the ClosedXML library that really simplifies the use of OpenXML to create xlsx spreadsheets.
Perhaps this Microsoft Article will help: How to: Open a spreadsheet document from a stream (Open XML SDK)
Related
let me outline my requirement. I have an excel spreadsheet with multiple pivot tables ( linked to charts / slicers etc ) and 2 worksheets with the data that those pivot tables refer to. Currently I have to manually execute a SQL query, copy the data, paste it over the current data in the spreadsheet and then refresh the pivot tables every day.
This is sub-optimal at best. So what I am trying to achieve is some C# code that I can execute on a schedule.
Using EPPlus, I have managed to load the excel file as a template, create a new one, get the data from SQL, update the 2 datasheets with the new data and then save the file.
using (var templateStream = new MemoryStream(File.ReadAllBytes(#"PATH_TO_TEMPLATE_FILE")))
{
using (var newStream = new MemoryStream())
{
//Create e NEW excel doc from the given template
using (ExcelPackage excelPackage = new ExcelPackage(newStream, templateStream))
{
//load the data from SQL
DataSet data = LoadDatasetFromQuery(configs, QueueItem);
//loop over the DataTables inside the DataSet
for (int i = 1; i <= data.Tables.Count; i++)
{
//Resolve the worksheet to put the data on
var worksheetName = configs.FirstOrDefault(c => c.Name.StartsWith($"Worksheet.{i}."));
ExcelWorksheet worksheet = excelPackage.Workbook.Worksheets[worksheetName.Value];
//Put the data on the worksheet top/left = B3
worksheet.Cells["B3"].LoadFromDataTable(data.Tables[i - 1], false);
}
//Save the file to the memory stream
excelPackage.Save();
}
//Write the file to the file system
File.WriteAllBytes(#"PATH_TO_OUTPUT_FILE", newStream.ToArray());
}
}
The problem is, when I try and open the excel file, it says it is corrupt and tries to repair it, which is does, by removing the pivot tables completely. My template file makes use of named ranges as referred to in this SO post but that has not resolved the issue.
Herewith the excel log of how it completed the "repair"
I have also dabbled a little bit in using the interop library ( Microsoft.Office.Interop.Excel ) but that is really like a black hole when it comes to debugging / documentation etc. I'm not averse to using it, I just don't know how. ( well nothing I have tried works properly anyways )
Any help with the above will be greatly appreciated. If you need more information, feel free to ask.
Ok, so it seems my above code was correct, but the excel template I was loading was dodgy. In order to correct the issue I had to make sure that all the pivot tables used named ranges to refer to the data ( click anywhere on the pivot table, then click on the Formulas tab in the top ribbon and then click on Name Manager ) source and then use the offset calculation ( to enable a dynamic range ) as suggested in the link in my post above.
=OFFSET(DataSource!$A$1,0,0,COUNTA(DataSource!$A:$A),COUNTA(DataSource!$1:$1))
where DataSource = the name of the worksheet with the data
Finally, I set up the pivots to refresh their data on opening ( right click on the pivot table, go to data tab and tick the "refresh on open" option )
There is a bit of a pain in that when I open the generated doc it is in "Protected mode" so the data + calcs dont refresh, but if I just click "Enable Editing" it all updates and normal service is resumed, happy days!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I'm trying to save an existing Excel file via ssms OR C# into my SQL Server 2016 database.
I want to save each row of my Excel file in a C# object and then save it into my database, or do you have better ideas?
I also thought about saving the Excel file as a *.csv and import this file via ssms in my database.
Which of these two ideas would you recommend or is there any other way to solve this problem?
If you have any questions, I would be pleased to answer them.
I thank you in advance for all the answers and tips!
For your problem you can try below approaches:
1) Using SQLBulkcopy:
SqlBulkCopy class as the name suggests does the bulk insert from one source to another and hence all rows from the Excel sheet can be easily read and inserted using the SqlBulkCopy class.
protected void Upload(object sender, EventArgs e)
{
//Upload and save the file
string excelPath = Server.MapPath("~/Files/") + Path.GetFileName(FileUpload1.PostedFile.FileName);
FileUpload1.SaveAs(excelPath);
string conString = string.Empty;
string extension = Path.GetExtension(FileUpload1.PostedFile.FileName);
switch (extension)
{
case ".xls": //Excel 97-03
conString = ConfigurationManager.ConnectionStrings["Excel03ConString"].ConnectionString;
break;
case ".xlsx": //Excel 07 or higher
conString = ConfigurationManager.ConnectionStrings["Excel07+ConString"].ConnectionString;
break;
}
conString = string.Format(conString, excelPath);
using (OleDbConnection excel_con = new OleDbConnection(conString))
{
excel_con.Open();
string sheet1 = excel_con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null).Rows[0]["TABLE_NAME"].ToString();
DataTable dtExcelData = new DataTable();
//[OPTIONAL]: It is recommended as otherwise the data will be considered as String by default.
dtExcelData.Columns.AddRange(new DataColumn[3] { new DataColumn("Id", typeof(int)),
new DataColumn("Name", typeof(string)),
new DataColumn("Salary",typeof(decimal)) });
using (OleDbDataAdapter oda = new OleDbDataAdapter("SELECT * FROM [" + sheet1 + "]", excel_con))
{
oda.Fill(dtExcelData);
}
excel_con.Close();
string consString = ConfigurationManager.ConnectionStrings["constr"].ConnectionString;
using (SqlConnection con = new SqlConnection(consString))
{
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(con))
{
//Set the database table name
sqlBulkCopy.DestinationTableName = "dbo.tblPersons";
//[OPTIONAL]: Map the Excel columns with that of the database table
sqlBulkCopy.ColumnMappings.Add("Id", "PersonId");
sqlBulkCopy.ColumnMappings.Add("Name", "Name");
sqlBulkCopy.ColumnMappings.Add("Salary", "Salary");
con.Open();
sqlBulkCopy.WriteToServer(dtExcelData);
con.Close();
}
}
}
}
Here this code adds an excel sheet with three columns as Id, Name and Salary.
2) Using DTS in SSMS:
You can use the SQL Server Data Transformation Services (DTS) Import Wizard or the SQL Server Import and Export Wizard to import Excel data into SQL Server tables. When you are stepping through the wizard and selecting the Excel source tables, remember that Excel object names that are appended with a dollar sign ($) represent worksheets (for example, Sheet1$), and that plain object names without the dollar sign represent Excel named ranges.
3) Using SSIS package:
You can create SSIS package to import excel file. For this, you can use BIDS in Visual Studio or SQL Server Data tools.
You can give your excel file as excel source and in the target give your SQL server database table.
Perform the necessary mappings and you're good to go.
Now, you must be having a question like When to use which approach?
Use approach 1, Whenever you're providing functionality to import excel file at the user end, i.e. according to application requirement, the user can upload local excel sheet. For this use case, one thing you should look out is, the user must be aware of the template. If you have written code to import excel with 3 columns and the user tries to import with 4 columns, you will have some error in future. So make sure that you provide a template that user should download and fill and upload it.
Use approach 2, whenever you want to load data for only one time, or you can say that you want to perform initial load. You can use this approach as it's most simple and requires less time to do the configuration.
Use approach 3, whenever you have some requirement like to import excel data on the timely basis from some shared location. For ex, you are importing monthly mobile bills to your database provided by some vendor. You can create a package for this functionality and do the SSIS configuration and create a package.
Once the package is created you can create a SQL job and schedule it as per the requirements.
You can use BulkInsert to imports a data file into a database table or view in a user-specified format in SQL Server
As all, it depends on usage, change frequency, who is going to maintain solution etc.
SSIS and CSV import
It is possible to create SSIS package which would be able to import your data automatically when deployed on MSSQL server or manually. This would be simplest/quickest to implement. One of advantages when using Visual Studio tooling for SSIS development you would have visual representation of mappings, flow.
Drawback, even though I have seen automated column mapping updates (C# automatic SSIS package generation), whenever you would need to add, remove, change column, you would need a manual change.
BCP
MS console utility which you can use to define columns in format files and import your CSVs. Drawback is that you there is no graphical user interface, though many would argue that this is an advantage because there is a better overview for changes.
ORM
In object relational mapping solution you would need to translate your Excel file into object oriented programming language classes and save as objects into database table. Drawback is that you need to have some programming knowledge, but would pay off in a longer run because potentially your solution could get the data directly form source for those excel sheets.
I am attempting to export rows of data from sql to excel but my Insert Command seems to fail every time. I have spent a good deal of time trying to create this but I have finally run up against the wall.
The excel document is one that is generated by the IRS and we are not aloud to modify anything above row 16. Row 16 is the header row, and everything below that needs to be the data from sql. The header names all have spaces in them, and that seems to be where I am running into trouble.
Starting at row 16 the column names are:
Attendee First Name, Attendee Last Name, Attendee PTIN, Program Number, CE Hours Awarded Program, Completion Date
This is how I am attempting to write to excel
private void GenerateReport()
{
FileInfo xlsFileInfo = new FileInfo(Server.MapPath(CE_REPORTS_PATH + CE_PTIN_TEMPLATE + EXTENSION));
string connectionString = String.Format(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties='Excel 8.0;HDR=Yes'", xlsFileInfo.FullName);
//create connection
OleDbConnection oleDBConnection = new OleDbConnection(connectionString);
oleDBConnection.Open();
//create the adapter with the select to get
OleDbDataAdapter adapter = new OleDbDataAdapter("SELECT * FROM [Sheet1$A16:F16]", oleDBConnection);
// Create the dataset and fill it by using the adapter.
DataTable dataTable = new DataTable();
adapter.FillSchema(dataTable, SchemaType.Source);
adapter.Fill(dataTable);
string[] colNames = new string[dataTable.Columns.Count];
string[] colParms = new string[dataTable.Columns.Count];
for (int i = 0; i < dataTable.Columns.Count; i++)
{
colNames[i] = String.Format("[{0}]", dataTable.Columns[i].ColumnName);
colParms[i] = "?";
}
// Create Insert Command
adapter.InsertCommand = new OleDbCommand(String.Format("INSERT INTO [Sheet1$] ({0}) values ({1})", string.Join(",", colNames), string.Join(",", colParms)), oleDBConnection);
// Create Paramaters
for (int i = 0; i < dataTable.Columns.Count; i++)
{
OleDbParameter param = new OleDbParameter(String.Format("#[{0}]", dataTable.Columns[i].ColumnName), OleDbType.Char, 255, dataTable.Columns[i].ColumnName);
adapter.InsertCommand.Parameters.Add(param);
}
// create a new row
DataRow newCERecord = dataTable.NewRow();
// populate row with test data
for (int i = 0; i < dataTable.Columns.Count; i++)
{
newCERecord[i] = "new Data";
}
dataTable.Rows.Add(newCERecord);
// Call update on the adapter to save all the changes to the dataset
adapter.Update(dataTable);
oleDBConnection.Close();
}
The error I get happens when adapter.Update(dataTable) is called and is as follows
$exception {"The INSERT INTO statement contains the following unknown field name: 'Attendee First Name'. Make sure you have typed the name correctly, and try the operation again."} System.Exception {System.Data.OleDb.OleDbException}
This is frustrating because I pull each field directly from the column name as gotten by colNames[i] = String.Format("[{0}]", dataTable.Columns[i].ColumnName). I discovered I needed the [] to account for the spaces in the column names, but at this point I am not sure what the problem is. When I look at the excel file everything seems correct to me.
I actually found a Microsoft article for you that has the entire code done - you can likely copy & paste whichever solution you like most. Here's the link:
http://support.microsoft.com/kb/306023
It seems like the one with CopyRecordset is your easiest approach, although they do explain the one I mentioned (using a tab-delimited file).
Edit: Here's my original answer for the sake of completeness. See the link above instead for more details and for a possible better solution.
This is not an answer to your question but a suggestion to change your approach (if you can). Excel tends to be very slow when adding data through COM calls and I assume OleDB uses COM internally. In my experience the fastest (and coincidentally the least painful way) to output data to Excel was to generate a tab-separated text file with all the data and then just import the file into Excel and use COM interop to perform any formatting on the sheet. When I generated Excel reports this way, most of my reports used to be generated almost 100x faster than using the Excel COM object model. (I don't know if this would be the same for OleDB calls, since I've never used OleDB with Excel but I'd be willing to bet the OleDB adapter uses COM internally.)
This would also take care of your embedded space problem since tab would be the column separator.
In your particular situation, I'd import the text file into Excel into a new sheet and then copy & paste it into the IRS sheet, at the right location. When done, the temporary sheet can be deleted.
I'm exporting a dataset to an Excel sheet, but I don't want to use the COM of Excel, because it takes a lot of time.
I need a method that exports to Excel without using the MS Office Interop, and I need to load the method using an empty Excel template so that the new Excel sheet has the same format.
You could reach an excel file and update its contents using ADO.NET and the Jet OleDbProvider
string con = #"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=your_path\test.xls;Extended Properties='Excel 8.0;HDR=No;'";
using(OleDbConnection c = new OleDbConnection(con))
{
c.Open();
string commandString = "Insert into [Sheet1$] (F1, F2, F3) values('F1Text', 'F2Text', 'F3Text')" ;
using(OleDbCommand cmd = new OleDbCommand(commandString))
{
cmd.Connection = c;
cmd.ExecuteNonQuery();
}
}
I use EPPlus. Available via Nuget and LGPL licensed. Let's you create and manage xlsx spreadsheets using OOXML.
I would look into something like this xlslinq
But you can also use this library as it will export it to a dataset
I would suggest using the Open XML SDK 2.0
You will be able to do everything you requested and on top of that it is very fast.
I have some tabular data that I'd like to turn into an Excel table.
Software available:
.NET 4 (C#)
Excel 2010 (using the Excel API is OK)
I prefer not to use any 3rd party libraries
Information about the data:
A couple million rows
5 columns, all strings (very simple and regular table structure)
In my script I'm currently using a nested List data structure but I can change that
Performance of the script is not critical
Searching online gives many results, and I'm confused whether I should use OleDb, ADO RecordSets, or something else. Some of these technologies seem like overkill for my scenario, and some seem like they might be obsolete.
What is the very simplest way to do this?
Edit: this is a one-time script I intend to run from my attended desktop.
Avoid using COM interop at all costs. Use a third-party API. Really. In fact, if you're doing this server-side, you virtually have to. There are plenty of free options. I highly recommend using EPPlus, but there are also enterprise-level solutions available. I've used EPPlus a fair amount, and it works great. Unlike interop, it allows you to generate Excel files without requiring Excel to be installed on the machine, which means you also don't have to worry about COM objects sticking around as background processes. Even with proper object disposal, the Excel processes don't always end.
http://epplus.codeplex.com/releases/view/42439
I know you said you want to avoid third-party libraries, but they really are the way to go. Microsoft does not recommend automating Office. It's really not meant to be automated anyway.
http://support.microsoft.com/kb/257757
However, you may want to reconsider inserting "a couple million rows" into a single spreadsheet.
Honoring your request to avoid 3rd party tools and using COM objects, here's how I'd do it.
Add reference to project: Com object
Microsoft Excel 11.0.
Top of module add:
using Microsoft.Office.Interop.Excel;
Add event logic like this:
private void DoThatExcelThing()
{
ApplicationClass myExcel;
try
{
myExcel = GetObject(,"Excel.Application")
}
catch (Exception ex)
{
myExcel = New ApplicationClass()
}
myExcel.Visible = true;
Workbook wb1 = myExcel.Workbooks.Add("");
Worksheet ws1 = (Worksheet)wb1.Worksheets[1];
//Read the connection string from App.Config
string strConn = System.Configuration.ConfigurationManager.ConnectionStrings["NewConnString"].ConnectionString;
//Open a connection to the database
SqlConnection myConn = new SqlConnection();
myConn.ConnectionString = strConn;
myConn.Open();
//Establish the query
SqlCommand myCmd = new SqlCommand("select * from employees", myConn);
SqlDataReader myRdr = myCmd.ExecuteReader();
//Read the data and put into the spreadsheet.
int j = 3;
while (myRdr.Read())
{
for (int i=0 ; i < myRdr.FieldCount; i++)
{
ws1.Cells[j, i+1] = myRdr[i].ToString();
}
j++;
}
//Populate the column names
for (int i = 0; i < myRdr.FieldCount ; i++)
{
ws1.Cells[2, i+1] = myRdr.GetName(i);
}
myRdr.Close();
myConn.Close();
//Add some formatting
Range rng1 = ws1.get_Range("A1", "H1");
rng1.Font.Bold = true;
rng1.Font.ColorIndex = 3;
rng1.HorizontalAlignment = XlHAlign.xlHAlignCenter;
Range rng2 = ws1.get_Range("A2", "H50");
rng2.WrapText = false;
rng2.EntireColumn.AutoFit();
//Add a header row
ws1.get_Range("A1", "H1").EntireRow.Insert(XlInsertShiftDirection.xlShiftDown, Missing.Value);
ws1.Cells[1, 1] = "Employee Contact List";
Range rng3 = ws1.get_Range("A1", "H1");
rng3.Merge(Missing.Value);
rng3.Font.Size = 16;
rng3.Font.ColorIndex = 3;
rng3.Font.Underline = true;
rng3.Font.Bold = true;
rng3.VerticalAlignment = XlVAlign.xlVAlignCenter;
//Save and close
string strFileName = String.Format("Employees{0}.xlsx", DateTime.Now.ToString("HHmmss"));
System.IO.File.Delete(strFileName);
wb1.SaveAs(strFileName, XlFileFormat.xlWorkbookDefault, Missing.Value, Missing.Value, Missing.Value, Missing.Value,
XlSaveAsAccessMode.xlExclusive, Missing.Value, false, Missing.Value, Missing.Value, Missing.Value);
myExcel.Quit();
}
Some things for your consideration...
If this is a client side solution, there is nothing wrong with using Interops.
If this is a server side solution, Don't use Interops. Good alternative is OpenXML SDK from Microsoft if you don't want 3rd party solution. It's free. I believe the latest one has similar object model that Excel has. It's a lot faster, A LOT, in generating the workbook vs going the interops way which can bog down your server.
I once read that the easiest way to create an Excel table was to actualy write a HTML table, including its structure and data, and simply name the file .xls.
Excel will be able to convert it, but it will display a warning saying that the content does not match the extension.
I agree that a 3rd party dll would be cleaner than the com, but if you go the interop route...
Hands down the best way to populate an excel sheet is to first put the data in a 2 dimensional string array, then get an excel range object with the same dimensions and set it (range.set_value2(oarray) I think). Using any other method is hideously slow.
Also be sure you use the appropriate cleanup code in your finally block.
i implemented "export to Excel" with the ms-access-ole-db-driver that can also read and write excel files the follwoing way:
preparation (done once)
create an excel file that contains all (header, Formatting, formulas, diagrams) with an empty data area as a template to be filled
give the data area (including the headers) a name (ie "MyData")
Implementing export
copy template file to destination folder
open an oledb-database connection to the destination file
use sql to insert data
Example
Excel table with Named area "MyData"
Name, FamilyName, Birthday
open System.Data.OleDb.OleDbConnection
execute sql "Insert into MyData(Name, FamilyName, Birthday) values(...)"
I used this connection string
private const string FORMAT_EXCEL_CONNECT =
// #"Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties=""Excel 8.0;HDR={1}""";
#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=""Excel 12.0;HDR={1}""";
private static string GetExcelConnectionString(string excelFilePath, bool header)
{
return string.Format(FORMAT_EXCEL_CONNECT,
excelFilePath,
(header) ? "Yes" : "No"
);
}