I am trying to import data from text file that is tab delimited using OleDbConnection string like below
using (OleDbConnection con =new OleDbConnection(#"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + path + #";Extended Properties='text;HDR=YES;FMT=TabDelimited';"))
I have provided tabDelimited option in extended properties but still i am getting data for each column upto comma ie CSV delimited.
I am importing data for the first time so this is kind of new to me.
I want to insert the data into datatable and then bulk insert the datatable into database (SQL Server) but the data is not received properly.
When using the ODBC text driver, you need to supply schema information as well.
See https://learn.microsoft.com/en-us/sql/odbc/microsoft/schema-ini-file-text-file-driver
Basically, you need to add a file named schema.ini in the same directory as your text file.
This should contain two rows:
[yourfilename.txt]
Format=TabDelimited
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I'm trying to save an existing Excel file via ssms OR C# into my SQL Server 2016 database.
I want to save each row of my Excel file in a C# object and then save it into my database, or do you have better ideas?
I also thought about saving the Excel file as a *.csv and import this file via ssms in my database.
Which of these two ideas would you recommend or is there any other way to solve this problem?
If you have any questions, I would be pleased to answer them.
I thank you in advance for all the answers and tips!
For your problem you can try below approaches:
1) Using SQLBulkcopy:
SqlBulkCopy class as the name suggests does the bulk insert from one source to another and hence all rows from the Excel sheet can be easily read and inserted using the SqlBulkCopy class.
protected void Upload(object sender, EventArgs e)
{
//Upload and save the file
string excelPath = Server.MapPath("~/Files/") + Path.GetFileName(FileUpload1.PostedFile.FileName);
FileUpload1.SaveAs(excelPath);
string conString = string.Empty;
string extension = Path.GetExtension(FileUpload1.PostedFile.FileName);
switch (extension)
{
case ".xls": //Excel 97-03
conString = ConfigurationManager.ConnectionStrings["Excel03ConString"].ConnectionString;
break;
case ".xlsx": //Excel 07 or higher
conString = ConfigurationManager.ConnectionStrings["Excel07+ConString"].ConnectionString;
break;
}
conString = string.Format(conString, excelPath);
using (OleDbConnection excel_con = new OleDbConnection(conString))
{
excel_con.Open();
string sheet1 = excel_con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null).Rows[0]["TABLE_NAME"].ToString();
DataTable dtExcelData = new DataTable();
//[OPTIONAL]: It is recommended as otherwise the data will be considered as String by default.
dtExcelData.Columns.AddRange(new DataColumn[3] { new DataColumn("Id", typeof(int)),
new DataColumn("Name", typeof(string)),
new DataColumn("Salary",typeof(decimal)) });
using (OleDbDataAdapter oda = new OleDbDataAdapter("SELECT * FROM [" + sheet1 + "]", excel_con))
{
oda.Fill(dtExcelData);
}
excel_con.Close();
string consString = ConfigurationManager.ConnectionStrings["constr"].ConnectionString;
using (SqlConnection con = new SqlConnection(consString))
{
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(con))
{
//Set the database table name
sqlBulkCopy.DestinationTableName = "dbo.tblPersons";
//[OPTIONAL]: Map the Excel columns with that of the database table
sqlBulkCopy.ColumnMappings.Add("Id", "PersonId");
sqlBulkCopy.ColumnMappings.Add("Name", "Name");
sqlBulkCopy.ColumnMappings.Add("Salary", "Salary");
con.Open();
sqlBulkCopy.WriteToServer(dtExcelData);
con.Close();
}
}
}
}
Here this code adds an excel sheet with three columns as Id, Name and Salary.
2) Using DTS in SSMS:
You can use the SQL Server Data Transformation Services (DTS) Import Wizard or the SQL Server Import and Export Wizard to import Excel data into SQL Server tables. When you are stepping through the wizard and selecting the Excel source tables, remember that Excel object names that are appended with a dollar sign ($) represent worksheets (for example, Sheet1$), and that plain object names without the dollar sign represent Excel named ranges.
3) Using SSIS package:
You can create SSIS package to import excel file. For this, you can use BIDS in Visual Studio or SQL Server Data tools.
You can give your excel file as excel source and in the target give your SQL server database table.
Perform the necessary mappings and you're good to go.
Now, you must be having a question like When to use which approach?
Use approach 1, Whenever you're providing functionality to import excel file at the user end, i.e. according to application requirement, the user can upload local excel sheet. For this use case, one thing you should look out is, the user must be aware of the template. If you have written code to import excel with 3 columns and the user tries to import with 4 columns, you will have some error in future. So make sure that you provide a template that user should download and fill and upload it.
Use approach 2, whenever you want to load data for only one time, or you can say that you want to perform initial load. You can use this approach as it's most simple and requires less time to do the configuration.
Use approach 3, whenever you have some requirement like to import excel data on the timely basis from some shared location. For ex, you are importing monthly mobile bills to your database provided by some vendor. You can create a package for this functionality and do the SSIS configuration and create a package.
Once the package is created you can create a SQL job and schedule it as per the requirements.
You can use BulkInsert to imports a data file into a database table or view in a user-specified format in SQL Server
As all, it depends on usage, change frequency, who is going to maintain solution etc.
SSIS and CSV import
It is possible to create SSIS package which would be able to import your data automatically when deployed on MSSQL server or manually. This would be simplest/quickest to implement. One of advantages when using Visual Studio tooling for SSIS development you would have visual representation of mappings, flow.
Drawback, even though I have seen automated column mapping updates (C# automatic SSIS package generation), whenever you would need to add, remove, change column, you would need a manual change.
BCP
MS console utility which you can use to define columns in format files and import your CSVs. Drawback is that you there is no graphical user interface, though many would argue that this is an advantage because there is a better overview for changes.
ORM
In object relational mapping solution you would need to translate your Excel file into object oriented programming language classes and save as objects into database table. Drawback is that you need to have some programming knowledge, but would pay off in a longer run because potentially your solution could get the data directly form source for those excel sheets.
I have a requirement to generate output report in Excel format and open the same on the screen when the processing is complete. But in this case, it should not save the report on the drive anywhere and only open on the screen.
I tried to use ADO using OLEDB but it always generates file before writing anything to it.
This is what I have tried so far.
using (OleDbConnection con = new OleDbConnection(connString))
{
try
{
con.Open();
}
catch (InvalidOperationException invalidEx)
{
//Exception handling
}
// Create table for excel structure
StringBuilder strSQL = new StringBuilder();
strSQL.Append("CREATE TABLE [" + tableName + "]([TITLE] text,[SURNAME] text,[STATUS] text)");
// Define file columns
StringBuilder strfield = new StringBuilder();
strfield.Append("[TITLE],[SURNAME],[STATUS]");
OleDbCommand cmd = new OleDbCommand(strSQL.ToString(), con);
cmd.ExecuteNonQuery(); // This creates the table
//Actual row for creating and insering row - logic not shown completely
cmd.CommandText = strSQL.Append(" insert into [" + tableName + "]( ")
.Append(strfield.ToString())
.Append(") values (").Append(strvalue).Append(")").ToString();
success = cmd.ExecuteNonQuery();
But this always creates the file first which I do not want.
Please advise if anyone has worked on the similar requirement. Thanks.
Ok, first off use ADO (a database access technology) to try and create a spreadsheet is bizarre, possibly doable, but definitely not easy.
Secondly you're saying create a spreadsheet and open it, without creating a file, this means that you'll also have to create ALL the functionality to open, parse, format and display spreadsheets (basically recreate Excel!)...as Excel cannot do this for you.
So I would question the "generate output report in Excel format" requirement, does this really mean "display in a grid"? Or is it "display in a grid that allows formatting, totalling?"
If it the Excel format really is a requirement, then the only thing I can suggest is you will have to create a temporary Excel file, then delete it after you've displayed it.
I would look at the ClosedXML library that really simplifies the use of OpenXML to create xlsx spreadsheets.
Perhaps this Microsoft Article will help: How to: Open a spreadsheet document from a stream (Open XML SDK)
I have a SQL Server 2008-R2 database that has a table that is storing different types of files (Word Docs, PDF, TIF, etc), I can successfully retrieve these files from the database using the following method:
private void GetFilesFromDatabase() {
try
{
const string connStr = #"Data Source=localhost\SQLInstance;Initial Catalog=MyData;Integrated Security=True;";
//Initialize SQL Server connection.
SqlConnection CN = new SqlConnection(connStr);
//Initialize SQL adapter.
SqlDataAdapter ADAP = new SqlDataAdapter("Select ole_id, ole_object From OLE Where ole_id = 21601", CN);
//Initialize Dataset.
DataSet DS = new DataSet();
//Fill dataset with FilesStore table.
ADAP.Fill(DS, "FilesStore");
//Get File data from dataset row.
byte[] FileData = (byte[])DS.Tables["FilesStore"].Rows[0]["ole_object"];
string FileName = #"C:\Temp\Text.doc";
//Write file data to selected file.
using (FileStream fs = new FileStream(FileName, FileMode.Create))
{
fs.Write(FileData, 0, FileData.Length);
fs.Close();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
This file I am retrieving is a Word Doc file according to other information contained in that particular row. When I attempt to open the file after retrieving it to disk the data all appears to be gibberish and is not able to be read. Now I believe that these files were compressed prior to be saved to the database but I don't know how to de-compress them so that they can be viewed, any thoughts on how I may accomplish this? My ultimate goal is to move these images into another database.
Your code to save files look OK. Assuming your original data was .DOC file and you got bad file after saving you pretty much out of luck. You may want to look at the content of the file in binary editor (i.e. Visual Studio's one) to confirm that file is not something obviously different (text/image....).
You need to ask around how files where stored in the database. There is no way this can be answered remotely as it could be compressed, encrypted, split into chunks or even simply corrupted.
I have a huge collection of Excel files. there are many information privite and my client want to store it in database. And later they can use the data in the database to rebuild all the Excel report. So do you have any idea to achieve that? what if I convert Excel to byte stream to store?
I know that if i put Excel to byte stream, will use more time and space to handle like formats and other thing, and stupid to do that. So I would like other way to store the data?
As Uriel_SVK said, Interop.Excel should be easy to achieve that. But if you just wish to store datas, can also have a try with Oledb.
string myConnection ="Provider=Microsoft.ACE.OLEDB.12.0;Data Source=D:\test.xlsx;Extended Properties="Excel 12.0 ;HDR=YES";
OleDbConnection conn = new OleDbConnection(connstr);
string strSQL = "SELECT * FROM [Sheet$]";
OleDbCommand cmd = new OleDbCommand(strSQL, conn);
DataSet dataset = new DataSet();
OleDbDataAdapter adapter = new OleDbDataAdapter(cmd);
adapter.Fill(dataset);
GridViewXYZ.DataSource = dataset;
GridViewXYZ.DataBind();
Are you constrained to using C#? Certain versions of SQL Server offer DTS or SSIS services for moving data in and out of the database from various sources/destinations such as Excel files. Oracle has something similar in OWB.
You can use Jet OleDB.
The sheet will be the tables and the workbook will be the database. You can use SQl query to produce the data what you want and save it on a datatable/dataset
Hello I'm trying to connect to a DBF and query it.
I use the following connectionstring:
string s = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + Path.GetDirectoryName(this.ShapePathFileName) + "\\;Extended Properties=dBASE IV;User ID=Admin;Password=;";
Then I want to fill a datatable it the result:
oleDbDataAdapter = new OleDbDataAdapter(selectCommand);
((DbDataAdapter)oleDbDataAdapter).Fill(dataTable);
But I've in the results values like this:
"ÒoþÚ"
but in the file I've this:
"ãoçé"
also if in the SQL statement I've WHERE name like '%é' I get no results, but with WHERE name like '%Ú' I've results
any ideas how to fix this?
Try adding
Collate=YourDbCollation
or
CodePage=YourCodePage
in the query string.
Even better than that, try donwloading an usign
VIsual FoxPro OleDB Provider
You have lot of samples of how the new query string should look like:
Visual Fox Pro Connection Strings