Develop Reference

Custom connection opening/closing for DbContext

EF Core opens and closes a DbConnection for each query by default, unless you pass in an already-open connection.
I have lots of small queries, so instead of opening and closing a connection each time, I'd like to keep the connection open for a period of five seconds at a time, while reusing that connection for each query/command. (The solution for the question linked above leave the connection open for the entire life of the DBContext.)
Putting aside the locking/concurrency issues, where can I inject custom connection resolving/opening logic in a DbContext? Something like
before executing query:
if connection is not open
open
set timer to fire close request in five seconds
take lock on connection (to prevent closing)
execute query
release lock

This is the standard way of doing transaction.
You can combine multiple queries.
using (var context = new SchoolContext())
{
var std = new Student()
{
FirstName = "Bill",
LastName = "Gates"
};
context.Students.Add(std);
// or
// context.Add<Student>(std);
context.SaveChanges();
std = context.Students.First<Student>();
std.FirstName = "Steve";
context.SaveChanges();
}
ef core can use same connection or different or based on connection pooling.
Ef core has connected and disconnected mode of transaction. I think this can suit you.
aving data in the disconnected scenario is a little bit different than in the connected scenario. In the disconnected scenario, the DbContext is not aware of disconnected entities because entities were added or modified out of the scope of the current DbContext instance. So, you need to attach the disconnected entities to a context with appropriate EntityState in order to perform CUD (Create, Update, Delete) operations to the database.
The following figure illustrates the CUD operations in disconnected scenario:
As per the above figure, disconnected entities (entities which are not being tracked by the DbContext) need to be attached to the DbContext with an appropriate EntityState. For example, Added state for new entities, Modified state for the edited entities and Deleted state for the deleted entities, which will result in an INSERT, UPDATE, or DELETE command in the database when the SaveChanges() method is called.
The following steps must be performed in order to insert, update or delete records into the DB table using Entity Framework Core in disconnected scenario:
Attach an entity to DbContext with an appropriate EntityState e.g. Added, Modified, or Deleted
Call SaveChanges() method
The following example demonstrates inserting a new record into the database using the above steps:
//Disconnected entity
var std = new Student(){ Name = "Bill" };
using (var context = new SchoolContext())
{
//1. Attach an entity to context with Added EntityState
context.Add<Student>(std);
//or the followings are also valid
// context.Students.Add(std);
// context.Entry<Student>(std).State = EntityState.Added;
// context.Attach<Student>(std);
//2. Calling SaveChanges to insert a new record into Students table
context.SaveChanges();
}
In the example above, std is a disconnected instance of the Student entity. The context.Add() method attaches a Student entity to a context with an Added state. The SaveChanges() method builds and executes the following INSERT statement:
exec sp_executesql N'SET NOCOUNT ON;
https://www.entityframeworktutorial.net/efcore/saving-data-in-disconnected-scenario-in-ef-core.aspx
These are important methods.
public DbContext(DbConnection existingConnection, bool contextOwnsConnection)
public DbContext(DbConnection existingConnection, DbCompiledModel model, bool contextOwnsConnection)
Behavior in EF6 and future versions
For EF6 and future versions we have taken the approach that if the calling code chooses to open the connection by calling context.Database.Connection.Open() then it has a good reason for doing so and the framework will assume that it wants control over opening and closing of the connection and will no longer close the connection automatically.
Note
This can potentially lead to connections which are open for a long time so use with care.
We also updated the code so that ObjectContext.Connection.State now keeps track of the state of the underlying connection correctly.
using System;
using System.Data;
using System.Data.Entity;
using System.Data.Entity.Core.EntityClient;
using System.Data.Entity.Infrastructure;
namespace ConnectionManagementExamples
{
internal class DatabaseOpenConnectionBehaviorEF6
{
public static void DatabaseOpenConnectionBehavior()
{
using (var context = new BloggingContext())
{
// At this point the underlying store connection is closed
context.Database.Connection.Open();
// Now the underlying store connection is open and the
// ObjectContext.Connection.State correctly reports open too
var blog = new Blog { /* Blog’s properties */ };
context.Blogs.Add(blog);
context.SaveChanges();
// The underlying store connection remains open for the next operation
blog = new Blog { /* Blog’s properties */ };
context.Blogs.Add(blog);
context.SaveChanges();
// The underlying store connection is still open
} // The context is disposed – so now the underlying store connection is closed
}
}
}
https://learn.microsoft.com/en-us/ef/ef6/fundamentals/connection-management?redirectedfrom=MSDN

In ADO.NET you can configure connection pooling to fit your requirements
A connection pool is created for each unique connection string. When a pool is created, multiple connection objects are created and added to the pool so that the minimum pool size requirement is satisfied. Connections are added to the pool as needed, up to the maximum pool size specified (100 is the default). Connections are released back into the pool when they are closed or disposed.
The connection pooler satisfies requests for connections by reallocating connections as they are released back into the pool. If the maximum pool size has been reached and no usable connection is available, the request is queued. The pooler then tries to reclaim any connections until the time-out is reached (the default is 15 seconds). If the pooler cannot satisfy the request before the connection times out, an exception is thrown.
The connection pooler removes a connection from the pool after it has been idle for approximately 4-8 minutes, or if the pooler detects that the connection with the server has been severed. Note that a severed connection can be detected only after attempting to communicate with the server. If a connection is found that is no longer connected to the server, it is marked as invalid. Invalid connections are removed from the connection pool only when they are closed or reclaimed.
If the SqlConnection goes out of scope, it won't be closed.
Therefore, you must explicitly close the connection by calling Close
or Dispose. Close and Dispose are functionally equivalent. If
the connection pooling value Pooling is set to true or yes, the
underlying connection is returned back to the connection pool. On the
other hand, if Pooling is set to false or no, the underlying
connection to the server is actually closed.
using (SqlConnection connection = new SqlConnection("Integrated Security=SSPI;Initial Catalog=Northwind"))
{
connection.Open();
// Pool A is created.
}
using (SqlConnection connection = new SqlConnection("Integrated Security=SSPI;Initial Catalog=pubs"))
{
connection.Open();
// Pool B is created because the connection strings differ.
}
using (SqlConnection connection = new SqlConnection("Integrated Security=SSPI;Initial Catalog=Northwind"))
{
connection.Open();
// The connection string matches pool A.
}
You can read more details in this article.
EF.Core and Connection pooling this
Related SO question. Entity Framework and Connection Pooling

How to implement single SQL transaction for multiple ObjectContexts in EF4

I have a fairly big database with tables created for different business modules.
We decided to create different edmx-files for different modules respectively.
However, how can I prevent the usage of MSDTC when trying to implement a TransactionScope for a logical action that will incur writing to multiple tables in different edmx? Again, the underlying database is the same, I wouldn't want to use MSDTC for this scenario.
Is there any way to pass in an opened SQL connection with active transaction?
Thanks for help in advance.
Regards,
William

TransactionScope enlists the MSDTC when the databases are different and/or the connection strings are different.
Rick Strahl has a great article on this (his perspective is LINQ to SQL, but it's applicable to EF). The money paragraphs:
TransactionScope is a high level Transaction wrapper that makes it
real easy to wrap any code into a transaction without having to track
transactions manually. Traditionally TransactionScope was a .NET
wrapper around the Distributed Transaction Coordinator (DTC) but it’s
functionality has expanded somewhat. One concern is that the DTC is
rather expensive in terms of resource usage and it requires that the
DTC service is actually running on the machine (yet another service
which is especially bothersome on a client installation).
However, recent updates to TransactionScope and the SQL Server Client
drivers make it possible to use TransactionScope class and the ease of
use it provides without requiring DTC as long as you are running
against a single database and with a single consistent connection
string. In the example above, since the transaction works with a
single instance of a DataContext, the transaction actually works
without involving DTC. This is in SQL Server 2008.
See also this SO question/answer where I found the link to Rick's blog.
So if you're connecting to the same database and are using the same connection string, the DTC should not be involved.

thanks for all replies above!
by the way, just managed to find a solution which is to use EntityConnection and EntityTransaction explicitly. A sample is like this:
string theSqlConnStr = "data source=TheSource;initial catalog=TheCatalog;persist security info=True;user id=TheUserId;password=ThePassword";
EntityConnectionStringBuilder theEntyConnectionBuilder = new EntityConnectionStringBuilder();
theEntyConnectionBuilder.Provider = "System.Data.SqlClient";
theEntyConnectionBuilder.ProviderConnectionString = theConnectionString;
theEntyConnectionBuilder.Metadata = #"res://*/";
using (EntityConnection theConnection = new EntityConnection(theEntyConnectionBuilder.ToString()))
{
theConnection.Open();
theET = null;
try
{
theET = theConnection.BeginTransaction();
DataEntities1 DE1 = new DataEntities1(theConnection);
//DE1 do somethings...
DataEntities2 DE2 = new DataEntities2(theConnection);
//DE2 do somethings...
DataEntities3 DE3 = new DataEntities3(theConnection);
//DE3 do somethings...
theET.Commit();
}
catch (Exception ex)
{
if (theET != null) { theET.Rollback(); }
}
finally
{
theConnection.Close();
}
}
with explicit use of EntityConnection & EntityTransaction, I can achieve the sharing of single connection and transaction for multiple ObjectContexts for a single database, yet without the need to incur the usage of MSDTC.
Hope this info is helpful. Gd luck!!

SqlConnection and the Pool, is keeping an open connection kung-foo or foo-bar?

I had thought I was clever. But in light of recent discoveries, I'm not so sure any more. During the page life cycle, there could by any number of database interactions. Some back to back, others spread out. So I invented an object that keeps an instance of an SQL connection alive in the HttpContext.Items dictionary. Every db request then uses this connection and when the http request has ended, I properly dispose of the connection. We're looking at a couple hundred milliseconds the connection would be open, and with some heavy http caching, running out of available connections isn't a concern.
The point was to prevent additional round trips due to the establishing of new connections. But when I stumbled upon the knowledge of connection pooling, I think it pretty invalidates the usefulness of preserving the SqlConnection. Or does it?
Is scenario A the same as Scenario B, performance wise? Which would you recommend? Does scenario B provide no performance gains, and even possibly hinders it because of some edge cases where the connection might not disposed of properly? Forgive the pseudo-ness in the examples, I don't want to clutter them with barf.
A
using (var connection = new SqlConnection(connectionString))
{
using (var command = new SqlCommand("...", connection))
{
... doing database stuff ...
}
}
... traversing the stack ...
using (var connection = new SqlConnection(connectionString))
{
using (var command = new SqlCommand("...", connection))
{
... doing database stuff ...
}
}
B
var connectionKeeper = new ConnectionKeeper();
// Add to the context items so it can be used anywhere
Context.Items.Add("Connection", connectionKeeper);
... traversing the stack ...
using (var command = new SqlCommand("...", connectionKeeper.Connection))
{
... doing database stuff
}
... traversing the stack ...
using (var command = new SqlCommand("...", connectionKeeper.Connection))
{
... doing database stuff
}
... traversing the stack ...
// The end of the request
sqlKeeper.Dispose();

Use your code from section A. Please let the connection pool do it's job. Avoid keeping a static SqlConnection around at all costs. The connection pool was designed for this.
Here's an MSDN article for your reference.
SQL Server Connection Pooling (ADO.NET)

No point in doing that in code unless you turn connection pooling off.
And you should have a real serious think before doing that, that's extreme circumstances.
Connection pooling was invented to address the situation you are trying to address with this "permanent" connection, so it will actually interfere with the built in optimisations and increase the volume, complexity, and fragility of your code.

How to rollback multiple Queries on different database servers in case of any error

I am using different SQL procedures in an application.
First procedures insert some rows then some processing in C#code and then 2nd procedure
do some updation then again some code processing then third procedure delete some record and then insert new record. When all is done on Sever 1 then data is fetch from this server and sent to Server 2 there record is deleted and new record is inserted.
IF there is error at any stage on any server in any procedure i want to roll back all the record.
I can not use begin trans because processing takes time and can not block table as others users are also using same tables in parallel. So kindly tell me how can i achieve it without blocking the table for other users.
Thanks in advance.
Edited (Added code example):
I tried Transaction Scope but i am getting exception while opening the connection. I configured MS DTC but may be not configured properly.
"
Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please enable DTC for network access in the security configuration for MSDTC using the Component Services Administrative tool."
using (TransactionScope ts = new TransactionScope(TransactionScopeOption.Required))
{
try
{
dl.SetBookReadyToLive(13570, false);
//SetBookReadyToLive
dl.AddTestSubmiitedTitleID(23402);
dl.AddBookAuthorAtLIve(13570, 1);
ts.Complete();
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
}
public void SetBookReadyToLive(long BookID, bool status)
{
try
{
if (dbConMeta.State != ConnectionState.Open)
dbConMeta.Open();
SqlCommand cmd = new SqlCommand("spSetBookReadyToLive", dbConMeta);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Clear();
cmd.Parameters.Add("#BookID", BookID);
cmd.Parameters.Add("#status", status);
cmd.ExecuteNonQuery();
if (dbConMeta.State == ConnectionState.Open)
dbConMeta.Close();
}
catch
{
if (dbConMeta.State == ConnectionState.Open)
dbConMeta.Close();
}
}
I get the exception on opening the connection of method>
I am using SQL Server 2000, i have set the configuration of MS DTC on the machine where SQL Server is installed and also on my PC from where i am running the code. But still same exception.
Kindly help me to configure it

You can use the TransactionScope class. It works generally well but in case of distributed SQL servers like in your case requires the MS DTC enabled in both servers and configured properly (security has to be granted for execution of network transactions, distributed ones and so on...)
here a copy paste from an example on MSDN, you could "almost" use it like this... :)
// Create the TransactionScope to execute the commands, guaranteeing
// that both commands can commit or roll back as a single unit of work.
using (TransactionScope scope = new TransactionScope())
{
using (SqlConnection connection1 = new SqlConnection(connectString1))
{
// Opening the connection automatically enlists it in the
// TransactionScope as a lightweight transaction.
connection1.Open();
// Create the SqlCommand object and execute the first command.
SqlCommand command1 = new SqlCommand(commandText1, connection1);
returnValue = command1.ExecuteNonQuery();
writer.WriteLine("Rows to be affected by command1: {0}", returnValue);
// If you get here, this means that command1 succeeded. By nesting
// the using block for connection2 inside that of connection1, you
// conserve server and network resources as connection2 is opened
// only when there is a chance that the transaction can commit.
using (SqlConnection connection2 = new SqlConnection(connectString2))
{
// The transaction is escalated to a full distributed
// transaction when connection2 is opened.
connection2.Open();
// Execute the second command in the second database.
returnValue = 0;
SqlCommand command2 = new SqlCommand(commandText2, connection2);
returnValue = command2.ExecuteNonQuery();
writer.WriteLine("Rows to be affected by command2: {0}", returnValue);
}
}
// The Complete method commits the transaction. If an exception has been thrown,
// Complete is not called and the transaction is rolled back.
scope.Complete();
}
source: TransactionScope Class
to minimize locks you could specify the IsolationLevel with the overload of the constructor which takes a TransactionScopeOptions, default is Serializable if you are fine with that you could set it to ReadCommitted.
Note: Personally I would not use this one unless absolutely needed, because it's a bit of a pain to have the DTC always configured and Distributed Transactions are in general slower than local ones but really depends on your BL / DAL logic.

Short answer : The same way you would do it if you would do it in MS SQL Management Studio.
You open a connection to a server.
Open a transaction for a specific server
You run your queries related to this server
You make sure to keep your connection alive while you... [go back to 1. for next server]
If all your queries worked, commit all your changes.
Else, rollback all your queries.
Warning : The first table will most likely be locked until you're done with all your servers/queries. What you could do here to help this : If you got a lot of data, you can transfer the data to temporary tables on every servers before doing the step #2. Once this is done, you open the transaction, do your fast things, then commit/rollback asap.
Note: I know you asked how to achieve this without locking the tables, hence why I added an idea in the « warning » part.

Query Multiple databases with single ado.net query

I have distributive DB architecture where data is stored in multiple SQL servers.
how can i do select/update/delete by running a single query. for example "select * from employees" should return data from all databases i have.
How can write single query which run across multiple SQL servers and gets a single consolidated view to my web server.
NOTE: Since the number of SQL servers may change at varied times so I am looking for something else than linked queries since managing the linked queries at scale( up or down) is a big pain

To talk to different databases / connections, you'll need a distributed transaction via TransactionScope; fortunately, this is actually easier than db-transactions (although you need a reference to System.Transactions.dll):
using(TransactionScope tran = new TransactionScope()) {
// lots of code talking to different databases / connections
tran.Complete();
}
Additionally, TransactionScope nest naturally, and SqlConnection enlists automatically, making it really easy to use.

Use TransactionScope.
If you open connections to different servers within the scope, the transaction will be escelated to a distributed transaction.
Example:
using (TransactionScope scope = new TransactionScope())
{
conn1.Open(); //Open connection to db1
conn2.Open(); //Open connection to db2
// Don't forget to commit the transaction so it won't rollback
scope.Complete()
}

You can't do what you're after with a single query unless you're willing to insert an intermediary of some kind, such as a SQL Express instance, that would mediate with the other servers, perhaps using SQL CLR. But that's messy.
It would be much easier to simply issue a bunch of async requests, and then merge the responses into a single DataTable (or equivalent) when they arrive. By using native ADO.NET style async calls, all queries can happen in parallel. You will of course need to use a lock while reading the data into a single DataTable.

The best solution here is to use a Virtual DBMS to blend your multiple back-ends into a single apparent backend -- so your query goes to the Virtual DBMS which then relays it appropriately to the actual data stores.
OpenLink Virtuoso is one option. Virtuoso opens connections to any ODBC-accessible (including JDBC-accessible, via an ODBC-to-JDBC Bridge) data source.
Your data consuming applications can connect to Virtuoso via ODBC, JDBC, OLE-DB, or ADO.NET as needed. All remote linked objects (Tables, Views, Stored Procedures, etc.) are available through all data access mechanisms.
While you can achieve similar results using the other techniques outlined here, those require the end user to know all about the back-end data structures, and to optimize queries themselves. With Virtuoso, a built-in Cost-based Optimizer will re-write queries to deliver the fastest possible results, with the least possible network traffic, based on the Virtual Schema constructed when you link in the remote objects.
Disclaimer: I work for OpenLink Software, but do not directly benefit from anyone choosing to use our products.

If you do not have access or privilegies to make a Linked server and generates View with consolidate JOINed Query with all Sql Servers, fill results from the same query statement at all Sql Server instances and make and make a union of results. Looping throwing all Databases connections and add collected data to a consolidated collection data structure, at this example I choice DataTable:
DataTable consolidatedEmployees = new DataTable();
foreach(ConnectionStringSettings cs in ConfigurationManager.ConnectionStrings)
{
consolidatedEmployees.Merge(
SelectTransaction("select * from employees", cs.ConnectionString));
}
Using this method example to query any SQL Server Database based on ADO.NET:
/// <summary>
/// Method to execute SQL Query statements with
/// Transaction scope using isolation level to select read commited data
/// </summary>
/// <param name="query">SQL Query statement</param>
/// <param name="connString">Connections String</param>
internal DataTable SelectTransaction(string query, string connString)
{
DataTable tableResult = null;
SqlCommand cmd = null;
SqlConnection conn = null;
SqlDataAdapter adapter = null;
TransactionOptions tranOpt = new TransactionOptions();
tranOpt.IsolationLevel = IsolationLevel.ReadCommitted;
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required, tranOpt))
{
tableResult = new DataTable();
try
{
conn = new SqlConnection(connString);
conn.Open();
cmd = new SqlCommand(query, conn);
adapter = new SqlDataAdapter(cmd);
adapter.Fill(tableResult);
break;
}
catch (Exception ex)
{
scope.Dispose();
throw new Exception("Erro durante a transação ao banco de Dados.", ex);
}
finally
{
if (null != adapter)
{
adapter.Dispose();
}
if (null != cmd)
{
cmd.Dispose();
}
if (null != conn)
{
conn.Close();
conn.Dispose();
}
}
scope.Complete();
}
return tableResult;
}
With this solution, only per attention to replicated data, needs after to make a distinct on consolidated result.