I have a simple problem with a not so simple solution... I am currently inserting some data into a database like this:
kompenzacijeDataSet.KompenzacijeRow kompenzacija = kompenzacijeDataSet.Kompenzacije.NewKompenzacijeRow();
kompenzacija.Datum = DateTime.Now;
kompenzacija.PodjetjeID = stranka.id;
kompenzacija.Znesek = Decimal.Parse(tbZnesek.Text);
kompenzacijeDataSet.Kompenzacije.Rows.Add(kompenzacija);
kompenzacijeDataSetTableAdapters.KompenzacijeTableAdapter kompTA = new kompenzacijeDataSetTableAdapters.KompenzacijeTableAdapter();
kompTA.Update(this.kompenzacijeDataSet.Kompenzacije);
this.currentKompenzacijaID = LastInsertID(kompTA.Connection);
The last line is important. Why do I supply a connection? Well there is a SQLite function called last_insert_rowid() that you can call and get the last insert ID. Problem is it is bound to a connection and .NET seems to be reopening and closing connections for every dataset operation. I thought getting the connection from a table adapter would change things. But it doesn't.
Would anyone know how to solve this? Maybe where to get a constant connection from? Or maybe something more elegant?
Thank you.
EDIT:
This is also a problem with transactions, I would need the same connection if I would want to use transactions, so that is also a problem...
Using C# (.net 4.0) with SQLite, the SQLiteConnection class has a property LastInsertRowId that equals the Primary Integer Key of the most recently inserted (or updated) element.
The rowID is returned if the table doesn't have a primary integer key (in this case the rowID is column is automatically created).
See https://www.sqlite.org/c3ref/last_insert_rowid.html for more.
As for wrapping multiple commands in a single transaction, any commands entered after the transaction begins and before it is committed are part of one transaction.
long rowID;
using (SQLiteConnection con = new SQLiteConnection([datasource])
{
SQLiteTransaction transaction = null;
transaction = con.BeginTransaction();
... [execute insert statement]
rowID = con.LastInsertRowId;
transaction.Commit()
}
select last_insert_rowid();
And you will need to execute it as a scalar query.
string sql = #"select last_insert_rowid()";
long lastId = (long)command.ExecuteScalar(sql); // Need to type-cast since `ExecuteScalar` returns an object.
last_insert_rowid() is part of the solution. It returns a row number, not the actual ID.
cmd = CNN.CreateCommand();
cmd.CommandText = "SELECT last_insert_rowid()";
object i = cmd.ExecuteScalar();
cmd.CommandText = "SELECT " + ID_Name + " FROM " + TableName + " WHERE rowid=" + i.ToString();
i = cmd.ExecuteScalar();
I'm using Microsoft.Data.Sqlite package and I do not see a LastInsertRowId property. But you don't have to create a second trip to database to get the last id. Instead, combine both sql statements into a single string.
string sql = #"
insert into MyTable values (null, #name);
select last_insert_rowid();";
using (var cmd = conn.CreateCommand()) {
cmd.CommandText = sql;
cmd.Parameters.Add("#name", SqliteType.Text).Value = "John";
int lastId = Convert.ToInt32(cmd.ExecuteScalar());
}
There seems to be answers to both Microsoft's reference and SQLite's reference and that is the reason some people are getting LastInsertRowId property to work and others aren't.
Personally I don't use an PK as it's just an alias for the rowid column. Using the rowid is around twice as fast as one that you create. If I have a TEXT column for a PK I still use rowid and just make the text column unique. (for SQLite 3 only. You need your own for v1 & v2 as vacuum will alter rowid numbers)
That said, the way to get the information from a record in the last insert is the code below. Since the function does a left join to itself I LIMIT it to 1 just for speed, even if you don't there will only be 1 record from the main SELECT statement.
SELECT my_primary_key_column FROM my_table
WHERE rowid in (SELECT last_insert_rowid() LIMIT 1);
The SQLiteConnection object has a property for that, so there is not need for additional query.
After INSERT you just my use LastInsertRowId property of your SQLiteConnection object that was used for INSERT command.
Type of LastInsertRowId property is Int64.
Off course, as you already now, for auto increment to work the primary key on table must be set to be AUTOINCREMENT field, which is another topic.
database = new SQLiteConnection(databasePath);
public int GetLastInsertId()
{
return (int)SQLite3.LastInsertRowid(database.Handle);
}
# How about just running 2x SQL statements together using Execute Scalar?
# Person is a object that has an Id and Name property
var connString = LoadConnectionString(); // get connection string
using (var conn = new SQLiteConnection(connString)) // connect to sqlite
{
// insert new record and get Id of inserted record
var sql = #"INSERT INTO People (Name) VALUES (#Name);
SELECT Id FROM People
ORDER BY Id DESC";
var lastId = conn.ExecuteScalar(sql, person);
}
In EF Core 5 you can get ID in the object itself without using any "last inserted".
For example:
var r = new SomeData() { Name = "New Row", ...};
dbContext.Add(r);
dbContext.SaveChanges();
Console.WriteLine(r.ID);
you would get new ID without thinking of using correct connection or thread-safety etc.
If you're using the Microsoft.Data.Sqlite package, it doesn't include a LastInsertRowId property in the SqliteConnection class, but you can still call the last_insert_rowid function by using the underlying SQLitePCL library. Here's an extension method:
using Microsoft.Data.Sqlite;
using SQLitePCL;
public static long GetLastInsertRowId(this SqliteConnection connection)
{
var handle = connection.Handle ?? throw new NullReferenceException("The connection is not open.");
return raw.sqlite3_last_insert_rowid(handle);
}
Related
I want to create simple database in runtime, fill it with data from internal resource and then read each record through loop. Previously I used LiteDb for that but I couldn't squeeze time anymore so
I choosed SQLite.
I think there are few things to improve I am not aware of.
Database creation process:
First step is to create table
using var create = transaction.Connection.CreateCommand();
create.CommandText = "CREATE TABLE tableName (Id TEXT PRIMARY KEY, Value TEXT) WITHOUT ROWID";
create.ExecuteNonQuery();
Next insert command is defined
var insert = transaction.Connection.CreateCommand();
insert.CommandText = "INSERT OR IGNORE INTO tableName VALUES (#Id, #Record)";
var idParam = insert.CreateParameter();
var valueParam = insert.CreateParameter();
idParam.ParameterName = "#" + IdColumn;
valueParam.ParameterName = "#" + ValueColumn;
insert.Parameters.Add(idParam);
insert.Parameters.Add(valueParam);
Through loop each value is inserted
idParameter.Value = key;
valueParameter.Value = value.ValueAsText;
insert.Parameters["#Id"] = idParameter;
insert.Parameters["#Value"] = valueParameter;
insert.ExecuteNonQuery();
Transaction commit transaction.Commit();
Create index
using var index = transaction.Connection.CreateCommand();
index.CommandText = "CREATE UNIQUE INDEX idx_tableName ON tableName(Id);";
index.ExecuteNonQuery();
And after that i perform milion selects (to retrieve single value):
using var command = _connection.CreateCommand();
command.CommandText = "SELECT Value FROM tableName WHERE Id = #id;";
var param = command.CreateParameter();
param.ParameterName = "#id";
param.Value = id;
command.Parameters.Add(param);
return command.ExecuteReader(CommandBehavior.SingleResult).ToString();
For all select's one connection is shared and never closed. Insert is quite fast (less then minute) but select's are very troublesome here. Is there a way to improve them?
Table is quite big (around ~2 milions records) and Value contains quite heavy serialized objects.
System.Data.SQLite provider is used and connection string contains this additional options: Version=3;Journal Mode=Off;Synchronous=off;
If you go for performance, you need to consider this: each independent SELECT command is a roundtrip to the DB with some extra costs. It's similar to a N+1 select problem in case of parent-child relations.
The best thing you can do is to get a LIST of items (values):
SELECT Value FROM tableName WHERE Id IN (1, 2, 3, 4, ...);
Here's a link on how to code that: https://www.mikesdotnetting.com/article/116/parameterized-in-clauses-with-ado-net-and-linq
You could have the select command not recreated for every Id but created once and only executed for every Id. From your code it seems every select is CreateCommand/CreateParameters and so on. See this for example: https://learn.microsoft.com/en-us/dotnet/api/system.data.idbcommand.prepare?view=net-5.0 - you run .Prepare() once and then only execute (they don't need to be NonQuery)
you could then try to see if you can be faster with ExecuteScalar and not having reader created for one data result, like so: https://learn.microsoft.com/en-us/dotnet/api/system.data.idbcommand.executescalar?view=net-5.0
If scalar will not prove to be faster then you could try to use .SingleRow instead of .SingleResult in your ExecuteReader for possible performance optimisations. According to this: https://learn.microsoft.com/en-us/dotnet/api/system.data.commandbehavior?view=net-5.0 it might work. I doubt that but if first two don't help, why not try it too.
SQL Server provides output for inserted and updated record with the 'inserted' keyword.
I have a table representing a processing queue. I use the following query to lock a record and get the ID of the locked record:
UPDATE TOP (1) GlobalTrans
SET LockDateTime = GETUTCDATE()
OUTPUT inserted.ID
WHERE LockDateTime IS NULL
This will output a column named ID with all the updated record IDs (a single ID in my case). How can I translate this into EF in C# to execute the update and get the ID back?
Entity Framework has no way of doing that.
You could do it the ORM way, by selecting all the records, setting their LockDateTime and writing them back. That probably is not safe for what you want to do because by default it's not one single transaction.
You can span your own transactions and use RepeatableRead as isolation level. That should work. Depending on what your database does in the background, it might be overkill though.
You could write the SQL by hand. That defeats the purpose of entity framework, but it should be just as safe as it was before as far as the locking mechanism is concerned.
You could also put it into a stored procedure and call that. It's a little bit better than the above version because at least somebody will compile it and check that the table and column names are correct.
Simple Example #1 to get a data table:
I did this directly against the connection:
Changed the command.ExecuteNonQuery() to command.ExecuteReader()
var connection = DbContext().Database.Connection as SqlConnection;
using (var command = connection.CreateCommand())
{
command.CommandText = sql;
command.CommandTimeout = 120;
command.Parameters.Add(param);
using (var reader = command.ExecuteReader())
{
var resultTable = new DataTable();
resultTable.Load(reader);
return resultTable;
}
}
FYI, If you don't have an OUTPUT clause in your SQL, it will return an empty data table.
Example #2 to return entities:
This is a bit more complicated but does work.
using a SQL statement with a OUTPUT inserted.*
var className = typeof(T).Name;
var container = ObjContext().MetadataWorkspace.GetEntityContainer(UnitOfWork.ObjContext().DefaultContainerName, DataSpace.CSpace);
var setName = (from meta in container.BaseEntitySets where meta.ElementType.Name == className select meta.Name).First();
var results = ObjContext().ExecuteStoreQuery<T>(sql, setName, trackingEnabled ? MergeOption.AppendOnly : MergeOption.NoTracking).ToList();
T being the entity being worked on
My access database had 700 records,every record had 50 fields。I use ODBC Query of PHP which query speed is very fast,but i use ODBC Query of C# ,it speed is very slowly,codes below:
m_conn = new OdbcConnection("DSN=real");//This DSN set by through the windows control panel,ODBC manager,system dsn
m_conn.Open();
string sqlstr="select * from table1 where id = 1";
OdbcCommand selectCMD = new OdbcCommand(sqlstr, m_conn);
OdbcDataReader myreader;
myreader = selectCMD.ExecuteReader();
if (myreader == null)
return null;
string s =myreader["field"].ToString();//here,execution speed is very slow,why?
thanks for help
Don't select more data for your application than you need to process.
Your statement select * from table1 where id = 1 is selecting all fields. If you only need the field named field, than change your select statement to select field from table1 where id = 1.
If you provide additional information on your database structure I may be able to be of more help.
There are a couple of suggestions here. I am guessing that you are accessing this code multiple times, which is where the slowness is coming in. This is most likely caused by you not disposing/closing of the connections properly. Below is the code refactored with the using clause, which enforces the use of Dispose after the call.
Also, instead of using the asterisk, specifying the name of the columns will help access optimize the query.
Finally, if you are only concerned about 1 variable, consider changing the query to only return the one value you are looking for and changing the call to an ExecuteScalar() call.
// Consider specifying the fields you are concerned with
string sqlstr="select * from table1 where id = 1";
using (var m_conn = new OdbcConnection("DSN=real"))//This DSN set by through the windows control panel,ODBC manager,system dsn
using (var selectCMD = new OdbcCommand(sqlstr, m_conn))
{
m_conn.Open();
using (var myreader= selectCMD.ExecuteReader())
{
if (myreader == null)
return null;
string s =myreader["field"].ToString();
}
}
I have a very big string to be updated to the memo field of FoxPro Table, I tried
cmd = db.GetSqlStringCommandWrapper("UPDATE xxx SET memo1 = "adfsd" WHERE condition1 = 'satisfied'");
db.ExecuteNonQuery(cmd);
This query overwrites the previous values in the memo1.
I Cannot use MODIFY memo in C#.
How do I append a string to a already existing memo field in Foxpro using C# ?
Try making the command say:
UPDATE xxx SET memo1 = memo1 + "adfsd"
I think the issue is probably with GetSqlStringCommandWrapper which as far as I can see is deprecated.
This shouldn't be a problem to do, for example using the OLEDB provider:
var DBC = #"C:\mydata.dbc";
string ConnectionString = string.Format("Provider=VFPOLEDB.1;Data Source={0};Exclusive=false;Ansi=true;OLE DB Services = 0", DBC);
using (OleDbConnection testConnection = new OleDbConnection(ConnectionString))
{
OleDbCommand updateCommand = new OleDbCommand(#"update mytable set mymemo=alltrim(mymemo)+ttoc(datetime()) where thisfield='THISVALUE'", testConnection);
testConnection.Open();
updateCommand.ExecuteNonQuery();
Console.WriteLine(#"Finished - press ENTER.");
Console.ReadLine();
}
You need to parameterize your query.. Assuming your query wrapper gets the sql connection handle to the database. The VFP OleDB Provider uses "?" as a "place-holder" for parameters and must match the order as associated to your query.
I have a more detailed sample to a very similar question here...
Try something like
string whatToSetItTo = "this is a test string that can even include 'quotes'";
cmd = db.GetSqlStringCommandWrapper("UPDATE YourTable SET memo1 = ? WHERE someKeyColumn = ?");
cmd.Parameters.Add( "parmForMemoField", whatToSetItTo);
cmd.Parameters.Add( "parmForKeyColumn", "satisfied" );
db.ExecuteNonQuery(cmd);
Notice the parameters added in same sequence. whatever the string value is (or could even be integer, date, etc respective to your table structure, but your sample only was based on strings) the place-holders are filled in order. The values would update accordingly.
I'm working on an ASP.NET project (C#) with SQL Server 2008.
When I insert a row into a table in the database, I would like to get the last inserted ID, which is the table's IDENTITY (Auto Incremented).
I do not wish to use another query, and do something like...
SELECT MAX(ID) FROM USERS;
Because - even though it's only one query - it feels lame...
When I insert something I usually use ExecuteNonQuery(), which returns the number of affected rows.
int y = Command.ExecuteNonQuery();
Isn't there a way to return the last inserted ID without using another query?
Most folks do this in the following way:
INSERT dbo.Users(Username)
VALUES('my new name');
SELECT NewID = SCOPE_IDENTITY();
(Or instead of a query, assigning that to a variable.)
So it's not really two queries against the table...
However there is also the following way:
INSERT dbo.Users(Username)
OUTPUT inserted.ID
VALUES('my new name');
You won't really be able to retrieve this with ExecuteNonQuery, though.
You can return the id as an output parameter from the stored procedure, e.g. #userId int output
Then, after the insert, SET #userId = scope_identity()
even though it's only one query - it feels lame...
It actually is also wrong as you can have multiple overlapping iserts.
That is one thing that I always fuind funny - people not reading the documentation.
SELECT SCOPE_IDENTITY()
returns the last identity value generated in a specific scope and is syntactically correct. It also is properly documented.
Isn't there a way to return the last inserted ID without using another query?
Yes. Ask for the number in the saame SQL batch.
INSERT (blablab9a); SELECT SCOPE_IDENTITY ();
as ONE string. ExecuteScalar.
You can have more than one SQL statement in one batch.
If you want to execute query from C# code & want to get last inserted id then you have to find the following code.
SqlConnection connection = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString);
connection.Open();
string sql = "Insert into [Order] (customer_id) values (" + Session["Customer_id"] + "); SELECT SCOPE_IDENTITY()";
SqlCommand cmd = new SqlCommand();
cmd.Connection = connection;
cmd.CommandText = sql;
cmd.CommandType = CommandType.Text;
var order_id = cmd.ExecuteScalar();
connection.Close();
Console.Write(order_id);