EF Core opens and closes a DbConnection for each query by default, unless you pass in an already-open connection.
I have lots of small queries, so instead of opening and closing a connection each time, I'd like to keep the connection open for a period of five seconds at a time, while reusing that connection for each query/command. (The solution for the question linked above leave the connection open for the entire life of the DBContext.)
Putting aside the locking/concurrency issues, where can I inject custom connection resolving/opening logic in a DbContext? Something like
before executing query:
if connection is not open
open
set timer to fire close request in five seconds
take lock on connection (to prevent closing)
execute query
release lock
This is the standard way of doing transaction.
You can combine multiple queries.
using (var context = new SchoolContext())
{
var std = new Student()
{
FirstName = "Bill",
LastName = "Gates"
};
context.Students.Add(std);
// or
// context.Add<Student>(std);
context.SaveChanges();
std = context.Students.First<Student>();
std.FirstName = "Steve";
context.SaveChanges();
}
ef core can use same connection or different or based on connection pooling.
Ef core has connected and disconnected mode of transaction. I think this can suit you.
aving data in the disconnected scenario is a little bit different than in the connected scenario. In the disconnected scenario, the DbContext is not aware of disconnected entities because entities were added or modified out of the scope of the current DbContext instance. So, you need to attach the disconnected entities to a context with appropriate EntityState in order to perform CUD (Create, Update, Delete) operations to the database.
The following figure illustrates the CUD operations in disconnected scenario:
As per the above figure, disconnected entities (entities which are not being tracked by the DbContext) need to be attached to the DbContext with an appropriate EntityState. For example, Added state for new entities, Modified state for the edited entities and Deleted state for the deleted entities, which will result in an INSERT, UPDATE, or DELETE command in the database when the SaveChanges() method is called.
The following steps must be performed in order to insert, update or delete records into the DB table using Entity Framework Core in disconnected scenario:
Attach an entity to DbContext with an appropriate EntityState e.g. Added, Modified, or Deleted
Call SaveChanges() method
The following example demonstrates inserting a new record into the database using the above steps:
//Disconnected entity
var std = new Student(){ Name = "Bill" };
using (var context = new SchoolContext())
{
//1. Attach an entity to context with Added EntityState
context.Add<Student>(std);
//or the followings are also valid
// context.Students.Add(std);
// context.Entry<Student>(std).State = EntityState.Added;
// context.Attach<Student>(std);
//2. Calling SaveChanges to insert a new record into Students table
context.SaveChanges();
}
In the example above, std is a disconnected instance of the Student entity. The context.Add() method attaches a Student entity to a context with an Added state. The SaveChanges() method builds and executes the following INSERT statement:
exec sp_executesql N'SET NOCOUNT ON;
https://www.entityframeworktutorial.net/efcore/saving-data-in-disconnected-scenario-in-ef-core.aspx
These are important methods.
public DbContext(DbConnection existingConnection, bool contextOwnsConnection)
public DbContext(DbConnection existingConnection, DbCompiledModel model, bool contextOwnsConnection)
Behavior in EF6 and future versions
For EF6 and future versions we have taken the approach that if the calling code chooses to open the connection by calling context.Database.Connection.Open() then it has a good reason for doing so and the framework will assume that it wants control over opening and closing of the connection and will no longer close the connection automatically.
Note
This can potentially lead to connections which are open for a long time so use with care.
We also updated the code so that ObjectContext.Connection.State now keeps track of the state of the underlying connection correctly.
using System;
using System.Data;
using System.Data.Entity;
using System.Data.Entity.Core.EntityClient;
using System.Data.Entity.Infrastructure;
namespace ConnectionManagementExamples
{
internal class DatabaseOpenConnectionBehaviorEF6
{
public static void DatabaseOpenConnectionBehavior()
{
using (var context = new BloggingContext())
{
// At this point the underlying store connection is closed
context.Database.Connection.Open();
// Now the underlying store connection is open and the
// ObjectContext.Connection.State correctly reports open too
var blog = new Blog { /* Blog’s properties */ };
context.Blogs.Add(blog);
context.SaveChanges();
// The underlying store connection remains open for the next operation
blog = new Blog { /* Blog’s properties */ };
context.Blogs.Add(blog);
context.SaveChanges();
// The underlying store connection is still open
} // The context is disposed – so now the underlying store connection is closed
}
}
}
https://learn.microsoft.com/en-us/ef/ef6/fundamentals/connection-management?redirectedfrom=MSDN
In ADO.NET you can configure connection pooling to fit your requirements
A connection pool is created for each unique connection string. When a pool is created, multiple connection objects are created and added to the pool so that the minimum pool size requirement is satisfied. Connections are added to the pool as needed, up to the maximum pool size specified (100 is the default). Connections are released back into the pool when they are closed or disposed.
The connection pooler satisfies requests for connections by reallocating connections as they are released back into the pool. If the maximum pool size has been reached and no usable connection is available, the request is queued. The pooler then tries to reclaim any connections until the time-out is reached (the default is 15 seconds). If the pooler cannot satisfy the request before the connection times out, an exception is thrown.
The connection pooler removes a connection from the pool after it has been idle for approximately 4-8 minutes, or if the pooler detects that the connection with the server has been severed. Note that a severed connection can be detected only after attempting to communicate with the server. If a connection is found that is no longer connected to the server, it is marked as invalid. Invalid connections are removed from the connection pool only when they are closed or reclaimed.
If the SqlConnection goes out of scope, it won't be closed.
Therefore, you must explicitly close the connection by calling Close
or Dispose. Close and Dispose are functionally equivalent. If
the connection pooling value Pooling is set to true or yes, the
underlying connection is returned back to the connection pool. On the
other hand, if Pooling is set to false or no, the underlying
connection to the server is actually closed.
using (SqlConnection connection = new SqlConnection("Integrated Security=SSPI;Initial Catalog=Northwind"))
{
connection.Open();
// Pool A is created.
}
using (SqlConnection connection = new SqlConnection("Integrated Security=SSPI;Initial Catalog=pubs"))
{
connection.Open();
// Pool B is created because the connection strings differ.
}
using (SqlConnection connection = new SqlConnection("Integrated Security=SSPI;Initial Catalog=Northwind"))
{
connection.Open();
// The connection string matches pool A.
}
You can read more details in this article.
EF.Core and Connection pooling this
Related SO question. Entity Framework and Connection Pooling
Related
I have an ASP.Net WebAPI instance setup that uses a MySQL database for storage. I have written an ActionFilter that handles creating a TransactionScope for the lifetime of a single endpoint request.
public async Task<HttpResponseMessage> ExecuteActionFilterAsync(
HttpActionContext actionContext,
CancellationToken cancellationToken,
Func<Task<HttpResponseMessage>> continuation)
{
var transactionScopeOptions = new TransactionOptions { IsolationLevel = IsolationLevel.ReadUncommitted };
using (var transaction = new TransactionScope(TransactionScopeOption.RequiresNew, transactionScopeOptions, TransactionScopeAsyncFlowOption.Enabled))
{
var handledTask = await continuation();
transaction.Complete();
return handledTask;
}
}
Then throughout the endpoints I have different queries/commands that open/close connections using the autoenlist=true functionality of DbConnection's. An example endpoint may be:
public async Task<IHttpActionResult> CreateStuffAsync()
{
var query = this.queryService.RetrieveAsync();
// logic to do stuff
var update = this.updateService.Update(query);
return this.Ok();
}
I don't create a single DbConnection and pass it around from the top, as this is a simplistic example, when in practise passing the connection between services would require a large refactor (although if necessary, this can be done). I also read that it is better to open/close the connections as necessary (i.e. keep them open for as little time as possible). The queryService and updateService open/close DbConnection's via using statements:
var factory = DbProviderFactories.GetFactory("MySql.Data.MySqlClient");
using (var connection = factory.CreateConnection())
{
connection.ConnectionString = "Data Source=localhost;Initial Catalog=MyDatabase;User ID=user;Password=password;Connect Timeout=300;AutoEnlist=true;";
if (connection.State != ConnectionState.Open)
{
connection.Open();
}
var result = await connection.QueryAsync(Sql).ConfigureAwait(false);
return result;
}
The same DbConnection is generally not used for multiple queries within the same API endpoint request -- but the same connection string is.
Intermittently I am seeing an exception thrown when attempting to open the connection:
"ExceptionType": "System.NotSupportedException",
"ExceptionMessage": "System.NotSupportedException: MySQL Connector/Net does not currently support distributed transactions.\r\n at MySql.Data.MySqlClient.ExceptionInterceptor.Throw(Exception exception)\r\n at MySql.Data.MySqlClient.MySqlConnection.EnlistTransaction(Transaction transaction)\r\n at MySql.Data.MySqlClient.MySqlConnection.Open()"
I do not understand why it is attempting to escalate the transaction to a distributed transaction, when all of the connections are against the same database. Or am I misunderstanding/misusing the TransactionScope and DbConnection instances?
The System.Transactions.Transaction object makes the determination of whether to escalate to a distributed transaction based on how many separate "resource managers" (e.g., a database) have enlisted in the transaction.
It does not draw a distinction between connections to different physical databases (that do require a distributed transaction) and multiple MySqlConnection connections that have the same connection string and connect to the same database (which might not). (It would be very difficult for it to determine that two separate "resource managers" ① represent the same physical DB and ② are being used sequentially, not in parallel.) As a result, when multiple MySqlConnection objects enlist in a transaction, it will always escalate to a distributed transaction.
When this happens, you run into MySQL bug #70587 that distributed transactions aren't supported in Connector/NET.
Workarounds would be:
Make sure only one MySqlConnection object is opened within any TransactionScope.
Change to a separate connector that does support distributed transactions. You could use MySqlConnector (NuGet, GitHub) as a drop-in replacement for Connector/NET. I've heard that dotConnect for MySQL supports them also (but haven't tried that one).
I know this question has been asked many times however none of the answers fit my issue.
I have a thread timer firing every 30 seconds that queries a MSSQL db that is under heavy load. If i need to update the data in the console app that i'm using i use Linq To Sql to update the data stored in memory.
My problem is sometimes I get the error ExecuteReader requires an open and available Connection.
The code from the thread timer fires a Thread.Run(reload());
The connection string is
//example code
void reload(...
string connstring = string.Format("Data Source={0},{1};Initial Catalog={2};User ID={3};Password={4};Application Name={5};Connect Timeout=120;MultipleActiveResultSets=True;Max Pool Size=1524;Pooling=true;"
settings = new ConnectionStringSettings("sqlServer", connstring, "System.Data.SqlClient");
using (var tx = new TransactionScope(TransactionScopeOption.Required,
new TransactionOptions() { IsolationLevel = IsolationLevel.ReadUncommitted }))
{
using (SwitchDataDataContext data = new SwitchDataDataContext(settings.ConnectionString))
{
data.CommandTimeout = 560;
then i do many linqtosql searches. The exceptions happen from time to time but not always on the same query's. it's like the connections is opened and is forced closed.
Sometimes the exceptions says the current status is Open, Closed, Connecting. I add a larger ThreadPool to the SQL db but nothing seems to help.
i also have ADO in other parts of the program without any issues.
I believe that your problem is that the transaction scope also has a timeout. The default timeout is 1 minute according to this answer. So the transaction times out long before your command does (560 seconds = 9.3 minutes or so) . You will need to set the timeout property in the instance of the TransactionOptions object you are creating
new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadUncommitted,
Timeout = new TimeSpan(0,10,0) /* 10 Minutes */
}
You can verify that is indeed the issue by setting the TransactionScope timeout to a small value to force it to timeout.
I changed Linq To Sql to Entity Framework and received the same type of message. I believe the issues is lazy Loading. I was using the collections before it was ready on a different thread. I just added .Include("Lab") to my collection to load the entire collection and it seems to of fixed the issue.
I'm creating an application where I connect to an Access database and do several updates. Since I'm not a database programmer this is also a learning experience. I found the following code online but it didn't work until I added the connection.open() line.
So here's my question. I thought that like in other using cases like creating a file it would automatically open the connection and then dispose at the last }. Why did I have to explicitly call out the open command after creating a new connection?
private static void GetAllTableAndColumnNames(string connectionString)
{
using (OdbcConnection connection =
new OdbcConnection(connectionString))
{
connection.Open();
DataTable tables = connection.GetSchema("Tables");
DataTable columns = connection.GetSchema("Columns");
foreach (DataRow row in columns.Rows)
{
Console.WriteLine(row["COLUMN_NAME"].ToString());
Console.WriteLine(row["TABLE_NAME"].ToString());
}
Console.Read();
}
}
Here's the error I got on runtime without the open command.
An unhandled exception of type 'System.InvalidOperationException' occurred in System.Data.dll
Additional information: Invalid operation. The connection is closed.
When the code reaches the end of a using block and it knows that the Connection object is about to be destroyed it "does you a favour" by ensuring that the connection is properly closed. That is because simply destroying the Connection object without notifying the server would be inconsiderate: server connections are often precious commodities and leaving an orphaned connection "open" on the server would be a Bad Thing.
On the other hand, automatically opening a connection when the Connection object is created (i.e., the using statement itself) would not necessarily be a Good Thing. Perhaps you want to create your Connection object, and then create a whole bunch of other objects that depend on the Connection object (e.g., Command, DataAdapter, etc.). Does the connection actually need to be open while that is taking place? If not, then having the connection open for the whole time might also be inconsiderate if you are connecting to a busy server.
Or, to put it another way, there are very legitimate reasons why a Connection object might switch states between "open" and "closed" while it exists, but the only state it should be in at the moment of its demise is "closed". That's why there is the "auto-close" behaviour but not a corresponding "auto-open".
Using clause is designed to close automaticlly resources when the variable is out of scope. The behaviour in the constructor depends on the implementation: DbConnection descendants don't open the conection in the constructor.
I think that this is because constructor overloads: some overloads doesn't take arguments, so can't open the database.
We have a web server which connects to a database using a single connection string, which makes it a strong candidate for being able to use a Connection Pool.
Do we need one SqlConnection object, or many?
i.e. Should we set up one connection in shared memory, and use that each time, or should we create a new one each time we want to use any connection object?
Is it the call to .Open() which assigns it from a pool, or the creating of a new object with the same connection string?
Also, do we need to call .Close() on the connection before it's released back into the pool, or is the variable going out of scope enough?
I read somewhere (I forget where exactly - sorry) that you shouldn't call close on connections in a pool, because it removes them from the pool somehow.
It’s worth bearing in mind that an identical connection string will re-use a connection which has been returned the the pool, but changing it in any way will cause a new connection to be made to the server.
i.e.
Assuming that SQLBox has IP 10.0.0.1
using (var conn = new SqlConnection(#"Server=10.0.0.1;...") {
conn.Open();
.. Do work ..
conn.Close();
}
using (var conn = new SqlConnection(#"Server=SQLBox;…") {
conn.Open(); // This will *NOT* reuse the connection from the pool.
.. Do work ..
conn.Close();
}
using (var conn = new SqlConnection(#"Server=10.0.0.1;...") {
conn.Open(); // This *WILL* reuse the connection from the pool.
.. Do work ..
conn.Close();
}
You should open a separate connection every time you need to access a database and close it once you're done with all the access. Do not keep your connections open for too long. It is not necessary and modern databases are definitely very good at handling multiple connections.
You can leave the management of the connection pool to SQL Server - it will do a good job as long as you'll not try to prevent it.
It's best to use local connection object and not share it across multiple parts of the code. You can use using pattern to make sure your connection will be closed:
using (SqlConnection conn = new SqlConnection("connectionstring"))
{
conn.Open();
// use your connection here
}
I am working on a unit of work class and I'm curious how the connection should be handled. My repositories take in a unit of work, and use that connection for Get() commands.
Obviously, Commit() will handle all Add, Updates and Deletes. This would open the connection and begin the transaction and close when finished. How should Gets be handled?
Should the UOW, open the connection in the constructor and close when completely finished? Meaning while I pass UOW from repo to repo, the connection is open. Or should I be opening and closing it only as needed?
Approach #1: Unit of work opens connection and connection remains open until processing is finished?
public UnitOfWork(IDbConnection connection)
{
Connection = connection;
Connection.Open();
Transaction = Connection.BeginTransaction();
}
Approach #2: Snippet of a Get method that opens right before read and closes right after. If passing to multiple repos, same connection is used, just opened and closed a bunch.
using (var reader = manager.GetReader())
{
UOW.Connection.Open();
while (reader.Read())
list.Add(factory.CreateTFromReader(reader));
UOW.Connection.Close();
}
No, you should always open the connection when needed and be absolutely sure to close it when the work is complete. The Connection Pooling mechanism will take care of keeping the connection available for your current user or for other users connecting concurrently on the same server
I think also that the second example is not quite right. You should have something like this
using (IDbConnection cn = manager.GetConnection)
using (var reader = manager.GetReader())
{
cn.Open();
while (reader.Read())
list.Add(factory.CreateTFromReader(reader));
}
This will ensure that the connection is closed even in the event of exceptions, and that is returned to the pool in order to be reused
Connection Pooling