I am trying to use SqlBulkCopy with .net core, but since I use a geometry column the following code requires Microsoft.SqlServer.Types which is not fully .net core compatible, especially on Linux.
using (SqlDataAdapter adapter = new SqlDataAdapter("SELECT TOP 0 * FROM " + tableName, sqlConnection))
{
DataTable dt = new DataTable();
adapter.Fill(dt);
return dt;
}
Without the dependency the Fill() fails as it cannot find the type.
Normally I use NetTopologySuite.IO.SqlServerBytes, but in this case the magic happens somewhere in the SqlDataAdapter and I don't know how to overwrite it.
I tried to create the DataTable columns manually without using Fill(), but it seems whatever type I specify I later get an error in the SqlBulkCopy.
The given value of type XYZ from the data source cannot be converted to type udt of the specified target column
I tried with SqlBytes and byte[], but nothing seems to work.
Update 1:
I got it working with a manually created DataTable with byte[] as the type for that column.
Still it would be nice to have a way using adapter.Fill(dt); or similar so that I do not have to manually list all the columns.
using (SqlDataAdapter da = new SqlDataAdapter("SELECT TOP 5 * FROM " + tableName, sqlConnection)
{
using (SqlCommandBuilder builder = new SqlCommandBuilder(da))
{
DataTable dt = new DataTable();
DataSet dataSet = new DataSet();
dt = dgPurchase.DataSource as DataTable;
//here assign the data table
da.Fill(dataSet, dt);
da.UpdateCommand = builder.GetUpdateCommand(true);
//da.InsertCommand = builder.GetInsertCommand(true);
//If new row then open it
da.Update(dataSet, dt);
}
}
// Can update though parameters .Follow the link
https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/updating-data-sources-with-dataadapters
//Useing inline code .Follow the link
http://csharp.net-informations.com/dataadapter/updatecommand-sqlserver.htm
Related
I am not understanding why I am getting an error after reading documentation in regards to the .Fill(). Am I missing something that is causing this to return with an error?
protected void FillData()
{
using (SqlConnection connection = new SqlConnection(#"Data Source = (LocalDB)\MSSQLLocalDB;AttachDbFilename= C:\Users\home\Documents\C# Programs\shop\Database.mdf ;Integrated Security = True"))
{
connection.Open();
using (SqlDataAdapter dataAdapter = new SqlDataAdapter("select * from Employee", connection))
{
DataTable table = new DataTable();
table.Fill(table);
employeeDataGridView.DataSource = table;
}
}
}
The problem is in this line of code
table.Fill(table);
You can't use table, to fill your table. The correct syntax would be
dataAdapter.Fill(table)
You can't populate a DataTable in that way, you need to fill your DataAdapter, use a DataSet and then set the DataGridView to use this.
using (SqlDataAdapter dataAdapter = new SqlDataAdapter("select * from Employee", connection))
{
var ds = new DataSet();
dataAdapter.Fill(ds);
employeeDataGridView.DataSource = ds.Tables[0];
}
The neatest way to code this up is:
protected void FillData()
{
using (SqlDataAdapter dataAdapter = new SqlDataAdapter("select * from Employee", #"Data Source = (LocalDB)\MSSQLLocalDB;AttachDbFilename= C:\Users\home\Documents\C# Programs\shop\Database.mdf ;Integrated Security=True"))
{
DataTable table = new DataTable();
dataAdapter.Fill(table);
employeeDataGridView.DataSource = table;
}
}
It'll get neater if you put your connection string in a static "global" variable somewhere
Points of note:
use the dataadapter constructor that takes two strings - it will create the connection and the command for you
you don't need to open the connection for the adapter- it knows how to
this means you can have just one using
if your sql needs parameters put them inside the using as dataAdapter.SelectCommand.Paramaters.Add...
you could turn this into a method that accepts any sql string and parameter collection and returns you a datatable
consider putting a WHERE clause in your sql; users might not like to see a grid with 20,000 employees in because it makes it harder to edit a handful of employees
It would be better to add a DataSet to your project (right click project, add>>new item, choose dataset - it gives you something that looks like a db visual design surface you can add queries to, create datatables that become components that can be added to your forms/create databound controls automatically) and create strongly typed datatables and table adapters
An alternative better route than this would be to use Dapper and strongly typed POCOs
Sorry for the vague title but I'm not sure how best to word it.
Basically, the issue that I'm having is that I initially was using an oledbconnection in C# to initiate a connection to a spreadsheet, query it, and load the results into a .net datatable. Unfortunately, it maxes out at 255 fields which I have around 600.
So, what I did instead was I created a dataset and tried to load 4 separate data tables with separate queries. Now for some reason, what's crazy to me is, if I load lets say, the first data table with 190 fields, and then I go on to query the spreadsheet again, if I go over that 250 mark (which I'd have 60 left), I get the following error:
An exception of type 'System.Data.OleDb.OleDbException' occurred in System.Data.dll but was not handled in user code
Additional information: No value given for one or more required parameters.
If I reduce the amount of fields in the second table to equal less than 250 though, it works. Anyway, is there a way that I can somehow clear the cache of the oledbconnection to somehow drop the first 190 fields or whatever is holding the result of the excel query so I can move on to the next? I tried doing a data adapter dispose but that didn't work, same issue. If I do a connection.dispose, I have to re-initialize the connection anyway. Is there a way I can still do this without having to drop the connection? Code below:
OleDbConnection cnn = new OleDbConnection(Settings.ExcelCN);
OleDbCommand fillT1 = new OleDbCommand(Settings.Excel_Proj_Select_1, cnn);
OleDbCommand fillT2 = new OleDbCommand(Settings.Excel_Proj_Select_2, cnn);
OleDbCommand fillT3 = new OleDbCommand(Settings.Excel_Proj_Select_3, cnn);
OleDbCommand fillT4 = new OleDbCommand(Settings.Excel_Proj_Select_4, cnn);
OleDbCommand updateCnt = new OleDbCommand(Settings.Excel_Update_Count_Select, cnn);
cnn.Open();
OleDbDataAdapter adp1 = new OleDbDataAdapter(fillT1);
OleDbDataAdapter adp2 = new OleDbDataAdapter(fillT2);
OleDbDataAdapter adp3 = new OleDbDataAdapter(fillT3);
OleDbDataAdapter adp4 = new OleDbDataAdapter(fillT4);
DataTable dt1 = new DataTable();
DataTable dt2 = new DataTable();
DataTable dt3 = new DataTable();
DataTable dt4 = new DataTable();
DataSet ds1 = new DataSet();
adp1.Fill(dt1);
ds1.Tables.Add(dt1);
adp2.Fill(dt2);
ds1.Tables.Add(dt2);
adp3.Fill(dt3);
ds1.Tables.Add(dt3);
adp4.Fill(dt4);
ds1.Tables.Add(dt4);
int rowcount = updateCnt.ExecuteNonQuery();
cnn.Close();
a question from a newbie to c# and apologies for the lenght of it. I have the following scenario. I have a small console application that populates a datatable by connecting to an external system and then needs to update existing records and insert new ones into an oracle table. The columns in the datatable are not named the same as the oracle table columns and not in the same order. I read another post on here with a similar scenario (loading from a file into a table) and it mentioned that doing an update/insert with an OracleDataAdapter would work. A simplified datatable and oracle table are
DataTable table = new DataTable();
table.Columns.Add("Product", typeof(String));
table.Columns.Add("Price", typeof(double));
table.Columns.Add("Effective_Date", typeof(DateTime));
//sample data
table.Rows.Add("abcd", 1.011, DateTime.Today);
table.Rows.Add("efg", 1.00, DateTime.Today);
table.Rows.Add("hijk", 20, DateTime.Today);
The oracle table has the structure
ITEM VARCHAR2(20 BYTE) NOT NULL ENABLE,
EFF_DATE DATE,
VALUE NUMBER
I have tried the following code to use the datatable and an adapter to update the oracle table but I'm missing something. I'm also wondering if I'm barking up the wrong tree. The majority of examples I have seen of using a dataadapter first does a select from the table and then puts the results into a grid where a user would be able to add, update, insert, or delete records and then uses the dataadapter to update the table. In my case I'm wondering if I get it to work if all records in the datatable will be treated as an insert anyway as there is no connection between the datatable and the oracle table.
I'm using the Oracle.ManagedDataAccess.Client to connect and do the updates
public static void UpdateOrSaveItems(DataTable dt)
{
String insert_statement, update_statement, select_statement;
select_statement = "SELECT * from items";
insert_statement = "INSERT INTO items (item, eff_date, value) values (:pInsItem,:pInsEffDate,:pInsValue)";
update_statement = "UPDATE items set eff_date = :pUpdEffDate, value = :pUpdValue where item = :pUpdItem";
using (OracleConnection conn = theDatabase.ConnectToDatabase())
{
using (OracleDataAdapter oraAdapter = new OracleDataAdapter(select_statement, conn))
{
//build update/insert commands and parameters
oraAdapter.UpdateCommand = new OracleCommand(update_statement, conn);
oraAdapter.InsertCommand = new OracleCommand(insert_statement, conn);
oraAdapter.UpdateCommand.BindByName = true;
oraAdapter.InsertCommand.BindByName = true;
OracleParameter pUpdItem = new OracleParameter("pUpdItem", OracleDbType.Varchar2);
pUpdItem.SourceColumn = dt.Columns[0].ColumnName;
OracleParameter pUpdEffDate = new OracleParameter("pUpdEffDate", OracleDbType.Date);
pUpdEffDate.SourceColumn = dt.Columns[2].ColumnName;
OracleParameter pUpdValue = new OracleParameter("pUpdValue", OracleDbType.Double);
pUpdValue.SourceColumn = dt.Columns[1].ColumnName;
OracleParameter pInsItem = new OracleParameter("pInsItem", OracleDbType.Varchar2);
pUpdItem.SourceColumn = dt.Columns[0].ColumnName;
OracleParameter pInsEffDate = new OracleParameter("pInsEffDate", OracleDbType.Date);
pInsEffDate.SourceColumn = dt.Columns[2].ColumnName;
OracleParameter pInsValue = new OracleParameter("pInsValue", OracleDbType.Double);
pInsValue.SourceColumn = dt.Columns[1].ColumnName; oraAdapter.UpdateCommand.Parameters.Add(pUpdItem);
oraAdapter.UpdateCommand.Parameters.Add(pUpdEffDate);
oraAdapter.UpdateCommand.Parameters.Add(pUpdValue);
oraAdapter.InsertCommand.Parameters.Add(pInsItem);
oraAdapter.InsertCommand.Parameters.Add(pInsEffDate);
oraAdapter.InsertCommand.Parameters.Add(pInsValue);
oraAdapter.Update(dt);
}
}
}
When I run this I get an error that I cannot insert a null into column that is defined as a key. In the datatable none of them are null. I'm missing something on telling it where the data is but am unsure what it is. Also wondering if this is the right way to do this sort of thing. I wanted to avoid
loop through datatable
select to see if record is in oracle table
if in table update else insert
because the volume of records could a couple of hundred thousand and wasn't sure what the performance would be like.
Are you initializing ColumnName properties of the DataTable object that you are passing in? If not, they could be reading as null.
For instance
public static void Main()
{
Datatable myDataTable = new DataTable();
myDataTable.Columns = new Columns[3];
myDataTable.Columns[0].ColumnName = "Employees";
myDataTable.Columns[1].ColumnName = "Salary";
myDataTable.Columns[2].ColumnName = "Department";
UpdateOrSaveItems(myDataTable);
}
I found the error. I hadn't set the source column on one of my insert parameters. I had set the source column on the pUdpItem twice instead of setting it for pUdpItem and pInsItem
Possible that I'm overlooking something here, but when I load a DataTable from SqlCommand.ExecuteReader() I am finding that the MaxLength property of my String field is ignored and reset to '50' in the resulting DataColumn. Here's an example table:
CREATE TABLE MySqlServerTable (
[instance_id] INT NOT NULL,
[field_id] INT NOT NULL,
[value] VARCHAR (MAX) NOT NULL);
and the method I'm using to initialize the local DataTable is
public DataTable GetDT()
{
string query = "SELECT top 0 * FROM MySqlServerTable;";
SqlCommand cmd = new SqlCommand(query, _msSqlConn);
DataTable dt = new DataTable();
dt.Load(cmd.ExecuteReader());
return dt;
}
where _msSqlConn is the already opened SqlConnection. If I then scroll through the DataColumns, I find that the value column (the string column) has been assigned a MaxLength of 50.
Console.WriteLine(GetDT().Columns["value"].MaxLength);
So what gives?
Kinda related to this attempted answer but still unresolved.
What's the right way to do this such that my string column MaxLengths are properly retrieved from the SqlServer2012 DB?
I think you need to use DataAdapter.FillSchema() to retrieve meta data. Try this something like this:
_msSqlConn.Open();
using (SqlDataAdapter da = new SqlDataAdapter(query, _msSqlConn))
{
DataTable dt = new DataTable();
da.FillSchema(dt, SchemaType.Mapped); //Or may be SchemaType.Source
return dt;
}
I don't use DataSets much. Usually find myself using an ORM or just a basic sqlReader.Read() followed by some GetValues(). I'm working on some legacy code that has DataSets all over the place, and while fixing a bug was trying to DRY some of it up.
However, I can't seem to actually get the data loaded into a non-typed DataSet.
public static DataSet ExecuteStoredProcedure(string storedProcedure, DBEnum db, IEnumerable<SqlParameter> parameters)
{
DataSet result = new DataSet();
using (SqlConnection connection = SqlHelper.GetSqlConnection(db))
{
SqlCommand command = connection.CreateCommand();
command.CommandType = CommandType.StoredProcedure;
command.CommandText = storedProcedure;
if (parameters != null)
foreach (SqlParameter parameter in parameters)
command.Parameters.Add(parameter);
connection.Open();
DataTable table = new DataTable();
using (SqlDataReader reader = command.ExecuteReader())
{
table.Load(reader);
}
result = table.DataSet; // table.DataSet is always empty!
}
return result;
}
I assumed table.Load(reader) does all the necessary reader.Read() calls ... but I went ahead and tried it both with and without reader.Read() before the table.Load(), to no avail.
I know that the stored procedure being called is actually returning data. If I do something like this, I see the data just fine:
using(SqlDataReader reader = command.ExecuteReader())
{
reader.Read();
object test = reader.GetValue(0); // returns the expected value
}
Seems like I'm missing something simple here, but I've been scratching my head over this one for a while now.
This is in .NET 3.5.
If you can, I would suggest using a SqlDataAdapter to populate the DataTable
using(SqlDataAdapter sqlDA = new SqlDataAdapter(command))
{
sqlDA.Fill(table);
}
You logic shows the DataTable being loaded with data from the reader but DataTable is never added to a dataset.
I believe the dataset should be created first. In fact, you could use DataSet.Load instead of the DataTable.Load. The DataSet.Load should create data tables, but it won't work the other way around.
A DataSet contains DataTables not the other way round. So if you want to create a DataSet to return then you'll probably want to create a new dataset and then add your DataTable into it before returning it. If a DataTable is not in a DataSet then the reference to its parent dataset will always be null.
That having been said I do also recommend Jason Evans' suggestion of using teh SqlDataAdapter.
Think of a dataset as a collection of tables and optionally information about the relationships between them.
In your example code you are creating an independent Table that does not belong to a DataSet. To pragmatically create a table that is part of a dataset you could do the following:
DataSet ds = new DataSet();
DataTable dt = ds.Tables.Add();
//or
ds.Tables.Add(MyAlreadyCreatedTable);
Jason Evans above is also correct, populating DataTables and DataSets is much simpler using SqlDataAdaptors as he demonstrated.
Finally, the Method as you have it written is meant to return a DataSet. But it only captures a single result set from the stored procedure it is calling. It's possible that a procedure could return any number of separate results.
All you should need to do is change the following:
DataTable table = new DataTable();
using (SqlDataReader reader = command.ExecuteReader())
{
table.Load(reader);
}
to
//you can skip creating a new DataTable object
using (SqlDataAdapter da = new SqlDataAdapter(command))
{
da.Fill(result); // the result set you created at the top
}