Develop Reference

How to read a CSV file into an SQL table

I'm trying to read a CSV file into a table that I have created in Visual Studio. I want to validate the values in the file to see if they are correct, if all the values pass the checks, it will go into the table. If any values are not correct, an error report will be created using a JSON file.
I have already got some test data ready but I'm not sure how to separate the correct data from the incorrect data after the checker are complete.
public partial class NHSBatchChecker : Form
{
public NHSBatchChecker()
{
InitializeComponent();
}
public void button1_Click(object sender, EventArgs e)
{
OpenFileDialog openFileDialog1 = new OpenFileDialog();
openFileDialog1.InitialDirectory = #"C:\Users\wy6282\Desktop\VS\NHSBATCHCHECKER\Test.txt"; // Start in C: drive
openFileDialog1.Title = "Browse Text Files";
openFileDialog1.RestoreDirectory = true;
openFileDialog1.DefaultExt = "txt"; // Extension of file is txt only
openFileDialog1.Filter = "Text|*.txt||*.*"; //Only text files allowed
openFileDialog1.CheckFileExists = true; // Error message if file does not exist
openFileDialog1.CheckPathExists = true; // Error message if invalid file path
if (openFileDialog1.ShowDialog() == DialogResult.OK)
{
string connectionstring;
SqlConnection cnn;
connectionstring = #"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=C:\Users\wy6282\Desktop\VS\NHSBATCHCHECKER\nhsBatchChecker\Results.mdf;Integrated Security=True";
cnn = new SqlConnection(connectionstring);
cnn.Open();
SqlCommand command;
SqlDataAdapter adaper = new SqlDataAdapter();
string sql = "";
sql = "Insert into Results(NHS Number, Date of Birth, First Name, Last Name, Title, Gender) values()";
command = new SqlCommand(sql, cnn);
adaper.InsertCommand = new SqlCommand(sql, cnn);
adaper.InsertCommand.ExecuteNonQuery();
command.Dispose();
cnn.Close();
How do I add my valid records into the sql table?

you're trying to do too much in one go.
rule number 1: always split your problem into manageable junks:
read data from CSV.
filter incorrect data
save filtered data to database.
Now you have 3 distinct pieces of work to focus on.
Reading data from a CSV is trivial, there are many libraries that can help with that. Do a bit of research and pick one.
Create a class which holds the properties you need for validation checks and also those you want saved in the database.
Your goal is to create a list of these objects, one per row in csv. Of course, you may not be able to read everything in one go depending on how much data your csv holds, but you can pick a library which can deal with whatever size you have.
Now you have a list of objects. Write an algorithm which determines what is valid and what not, based on the rules you need. Your goal here is to end up with a possibly smaller list of the same objects, chucking out the invalid ones.
Save whatever is left in the database.
You need to start thinking about how you organize you code, don't just throw everything into the Click event of a button. You can create a model class to hold your objects, maybe create a separate class library where you can put your csv reading method.
Another class library perhaps for your filtering algorithm(s).
Your Click event should be fairly slim and only call library methods when it needs to do something. This is separation of concerns or SOC, which is a Solid Principle.

I am not sure how you plan to validate the datapoints, but the code below shows how to pull data from a CSV and load it into a SQL Server table.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Data.SqlClient;
using System.Data.OleDb;
using System.Configuration;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
string server = "EXCEL-PC\\EXCELDEVELOPER";
string database = "AdventureWorksLT2012";
string SQLServerConnectionString = String.Format("Data Source={0};Initial Catalog={1};Integrated Security=SSPI", server, database);
string CSVpath = #"C:\Users\Ryan\Documents\Visual Studio 2010\Projects\Bulk Copy from CSV to SQL Server Table\WindowsFormsApplication1\bin"; // CSV file Path
string CSVFileConnectionString = String.Format("Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};;Extended Properties=\"text;HDR=Yes;FMT=Delimited\";", CSVpath);
var AllFiles = new DirectoryInfo(CSVpath).GetFiles("*.CSV");
string File_Name = string.Empty;
foreach (var file in AllFiles)
{
try
{
DataTable dt = new DataTable();
using (OleDbConnection con = new OleDbConnection(CSVFileConnectionString))
{
con.Open();
var csvQuery = string.Format("select * from [{0}]", file.Name);
using (OleDbDataAdapter da = new OleDbDataAdapter(csvQuery, con))
{
da.Fill(dt);
}
}
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(SQLServerConnectionString))
{
bulkCopy.ColumnMappings.Add(0, "MyGroup");
bulkCopy.ColumnMappings.Add(1, "ID");
bulkCopy.ColumnMappings.Add(2, "Name");
bulkCopy.ColumnMappings.Add(3, "Address");
bulkCopy.ColumnMappings.Add(4, "Country");
bulkCopy.DestinationTableName = "AllEmployees";
bulkCopy.BatchSize = 0;
bulkCopy.WriteToServer(dt);
bulkCopy.Close();
}
}
catch(Exception ex)
{
MessageBox.Show(ex.Message, "Warning!", MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
}
}
}
}
}

Unrecognized database format

I would like to ask, why I get this exception when I try to connect excel 2000/3 and also 2010?
using System;
using System.Collections.Generic;
using System.Text;
using System.Data.OleDb;
namespace md1_connect
{
class Program
{
static void Main (string[] args)
{
string ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=\"Book1.xls\"";
OleDbConnection MyConn = new OleDbConnection(ConnectionString);
OleDbCommand cmd = new OleDbCommand("SELECT * FROM[Sheet2$]", MyConn);
MyConn.Open();
OleDbDataReader dataReader = cmd.ExecuteReader();
while (dataReader.Read())
{
Console.WriteLine(dataReader.GetDouble(0));
}
MyConn.Close();
}
}
}

You need to tell the provider you're using Excel 97-2003 (xls as opposed to xlsx) by appending:
Extended Properties="Excel 8.0"
E.g.
string ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=\"Book1.xls\";Extended Properties=\"Excel 8.0\"";

I dont know what the exception is but I may know what you are talking about. You are probably compiling as x64 bit, please force it to run as 32 bit (x86). I believe that setting can be set in your Project Properties under Build Options

C# Read Excel data that is linked to Oracle DB

How do you read data from an Excel.xslx that is linked to Oracle? I can read the entire sheet just fine using C# and print to the console window, however this data that is being pulled (and refreshed every so often) cannot be read as a named range or table. I have tested reading data as named ranges/tables in other workbooks (not being pulled from Oracle) just fine. I would prefer not to read the entire sheet. So my question is how i would refer to this linked data as a named range or table in my sql query using C#, or if there is some other way to just read that linked data from Excel? Code is posted below followed by the output error (which has no build warnings/errors), 'PersonTable' is the named range i gave to the data that is being pulled from oracle11g.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.OleDb;
namespace ReadExcelNamedRage
{
class Program
{
static void Main(string[] args)
{
String connectionString = "Provider=Microsoft.Ace.OleDb.12.0; data source=c:\\documents and settings\\jhamandi\\desktop\\hrOracleTest.xlsx;extended properties=Excel 8.0";
//Select using a Named Range
string selectString = "SELECT * FROM PersonTable";
//Select using a Worksheet name
// string selectString = "SELECT * FROM [Sheet1$]";
OleDbConnection con = new OleDbConnection(connectionString);
OleDbCommand cmd = new OleDbCommand(selectString, con);
try
{
con.Open();
Console.WriteLine("Connection opened");
OleDbDataReader theData = cmd.ExecuteReader();
while (theData.Read())
{
Console.WriteLine("{0}: {1} ({2}) - {3} ({4})", theData.GetDouble(0),
theData.GetString(1), theData.GetString(2), theData.GetString(3), theData.GetString(4));
// Console.WriteLine("Writing");
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
con.Dispose();
Console.WriteLine("Connection closed");
}
}
}
}
Connection opened
The Microsoft Access database engine could not find the object PersonTable. Ma
ke sure the object exists and that you spell its name and the path name correctl
y. If PersonTable is not a local object, check your network connection or cont
act the server administrator.
Connection closed
Press any key to continue . . .

WPF App run SQL Server Scripts

I often have to 'Generate Scripts' for stored procedures on SQL Server Management Studio. I need to create a WPF app that will take the file produced and execute it. I know you can do this:
SqlCommand insert = new SqlCommand(sql, conn);
insert.ExecuteNonQuery();
If I pass the entire scripts file text into sql, will this execute everything?

Look at this: How to execute an .SQL script file using c#
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.SqlServer.Management.Smo;
using Microsoft.SqlServer.Management.Common;
using System.IO;
using System.Data.SqlClient;
[...]
private void Execute(object sender, EventArgs e)
{
string sqlConnectionString = #"Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=ccwebgrity;Data Source=SURAJIT\SQLEXPRESS";
FileInfo file = new FileInfo(#"E:\Project Docs\MX462-PD\MX756_ModMappings1.sql");
string script = file.OpenText().ReadToEnd();
SqlConnection conn = new SqlConnection(sqlConnectionString);
Server server = new Server(new ServerConnection(conn));
server.ConnectionContext.ExecuteNonQuery(script);
file.OpenText().Close();
}

Where/how do I write T-SQL in my C# code and get the result (Visual Studio 2008)

I am trying to write t-sql in C# (visual studio). I have this code to connect to the database:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Data.SqlClient;
using Microsoft.SqlServer.Server;
namespace WindowsFormsApplication8
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
string connetionString = null;
SqlConnection cnn;
connetionString = "Data Source=.\\SQLEXPRESS;AttachDbFilename=C:\\Users\\Xtreme\\Desktop\\CardsDB.mdf;Integrated Security=True;Connect Timeout=30;User Instance=True";
cnn = new SqlConnection(connetionString);
try
{
cnn.Open();
MessageBox.Show("Connection Open ! ");
cnn.Close();
}
catch (Exception ex)
{
MessageBox.Show("Can not open connection ! ");
}
}
}
}
Where/how do I write the T-SQL code and how do I get the result?
Can someone give me an simple select example in my code?

You can use DataAdapter.Fill Method:
try
{
using (SqlDataAdapter a = new SqlDataAdapter("SELECT * FROM Employee", cnn))
{
// Use DataAdapter to fill DataTable
DataTable t = new DataTable();
a.Fill(t);
// Render data onto the screen
dataGridView1.DataSource = t; //if you want.
}
}
catch (Exception ex)
{
MessageBox.Show("Problem!");
}

Create a SqlCommand and set the CommandType to CommandType.Text. Then add your SQL to the CommandText property of the SqlCommand.
SqlCommand command = new SqlCommand(commandName, (SqlConnection)Connection);
command.CommandType = CommandType.Text;
command.CommandText = "SELECT * FROM MyTable";
IDataReader result = command.ExecuteReader();

Ardman already showed you how to execute any arbitary sql command. But what i didn't answer very well, is where to put your sql statement.
I think it is a very bad behaviour and also not very good to read if you put your statement directly into the code.
A better method (in my eyes) is the following:
Within your project create a new folder (maybe called Queries)
Right click this folder and select Add - New Item
In the dialog just select Textfile and give it the name about what this query will do
Make sure you replace the file extension from .txt to .sql
Just put your statement right into this file
In the Resource Editor add this file as a resource
Now you can access this sql statement within your project just by using Properties.Resources.MySqlStatement