Suppose I have such pseudo code using some pseudo ORM (ok in my case it's Linq2Db).
static IEnumerable<A> GetA()
{
using (var db = ConnectionFactory.Current.GetDBConnection())
{
return from a in db.A
select a;
}
}
static B[] DoSmth()
{
var aItems = GetA();
if (!aItems.Any())
return null;
return aItems.Select(a => new B(a.prop1)).ToArray();
}
When will Connection be closed in db? Would it be closed at all in that case? What connection would be closed - those in using statement or those in lambda expression? .NET compiler is creating anonymous class for lambdas, so it will copy connection to that class. When would that connection be closed?
Somehow I managed to get Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. I materialized queries and exception disappeared. But I'm wondering how this thing works.
This is entirely dependent on your ORM. If disposing of ConnectionFactory.Current.GetDBConnection() closes the connection then you will never be able to enumerate the result. If it doesnt close the connection (and something else does) it might work depending on if someone else has closed the connection.
In any case you probably dont want to return an un-enumerated enumerable from something which creates and disposes the connection.
either enumerate the collection before closing it eg:
static IEnumerable<A> GetA()
{
using (var db = ConnectionFactory.Current.GetDBConnection())
{
return (from a in db.A
select a).ToArray();
}
}
or control the connection at the level which does enumerate the results eg:
static IEnumerable<A> GetA(whatevertype db)
{
return from a in db.A
select a;
}
static B[] DoSmth()
{
using (var db = ConnectionFactory.Current.GetDBConnection())
{
var aItems = GetA(db);
if (!aItems.Any())
return null;
return aItems.Select(a => new B(a.prop1)).ToArray();
}
}
Linq2db release connection when you close your connection object. In your example it happens when you leave using block and db object is disposed. This means that you cannot return query from connection scope - you need to fetch your data first or postpone connection disposal till you didn't get all required data.
Exception you observed was due to error in linq2db prior to 1.8.0 where it allowed query execution on disposed connection, which lead to connection leak. See https://github.com/linq2db/linq2db/issues/445 for more details. After that fix you cannot write code like GetA() in your example as you will get ObjectDisposedException when you will try to enumerate results.
after first call :
in this call connection opened in using scope , without any fetching data at the end of usnig scope Closed.
var aItems = GetA();
but at this line :
if (!aItems.Any())
there isn't any open connection
Related
Here is a very simplistic example of my current state:
using(var scope = new TransactionScope(TransactionScopeOption.Required, transactionScopeTimeout, TransactionScopeAsyncFlowOption.Enabled))
{
const string ConnectionString="server=localhost;username=;password=;enlist=false;Initial Catalog=blubber"
using(var myContext = new MyContext(ConnectionString))
{
var myTestObject = new TestObjectBuilder().WithId(1).Build();
myContext.Add(myTestObject);
myContext.SaveChanges();
}
// ...
using(var myContext = new MyContext(ConnectionString))
{
var myObj = myContext.MyObjects.Single(s => s.Id == 1); // This is the same object as above
}
}
I have a TransactionScope which will be used here, but in my connectionstring I explicitly say I don't wanna enlist my Entity Framework (6.2) Transactions.
The current behavior is that on the Single(s => s.Id == 1) Expression I get an error after 30 seconds (the default timeout) that the element could not be found.
First of all: Why do I not get a timeout-Exception or any SqlException? Second: Why is my data-row locked in the database?
Also via Sql Server Management Studio I can not query that exact row (only with the NOLOCK hint).
If I remove the TransactionScope or set the ISOLATION LEVEL to READ UNCOMMITED before the Single query everything works fine.
Also because of the Dispose of the transaction-scope at the end all data will be removed / rollbacked which also should not happen if I didn't enlist to this AmbientTransaction.
So my expected behavior is, that I get no lock and my data is persisted even the transactionscope is disposed and rollbacked. EF should ignore the transactionscope here.
Do I miss something critical here?
EDIT: I tried the same thing with NHibernate and here it works like a charm.
Please give expert opinion, refer to below static sorted list based on key value pair.
Method1 for close connection uses approach of accessing sorted list using key.
Method2 for close connection uses lock statement on the Sorted List and access it by index.
Please guide which approach is better as thousands of users simultaneously creating thousands of connections on web application. Note, accessing by index without locking can raise Index out of bound exception.
internal class ConnA
{
static internal SortedList slCons = new SortedList();
internal static bool CreateCon(string ConnID)
{
string constring = "sqlconnectionstring_containing_DataSource_UserInfo_InitialCatalog";
SqlConnection objSqlCon = new SqlConnection(constring);
objSqlCon.Open();
bool connSuccess = (objSqlCon.State == ConnectionState.Open) ? true : false;
if (connSuccess && slCons.ContainsKey(ConnID) == false)
{
slCons.Add(ConnID, objSqlCon);
}
return connSuccess;
}
//Method1
internal static void CloseConnection(string ConnID)
{
if (slCons.ContainsKey(ConnID))
{
SqlConnection objSqlCon = slCons[ConnID] as SqlConnection;
objSqlCon.Close();
objSqlCon.Dispose();
objSqlCon.ResetStatistics();
slCons.Remove(ConnID);
}
}
//Method2
internal static void CloseConnection(string ConnID)
{
lock (slCons)
{
int nIndex = slCons.IndexOfKey(ConnID);
if (nIndex != -1)
{
SqlConnection objSqlCon = (SqlConnection)slCons.GetByIndex(nIndex);
objSqlCon.Close();
objSqlCon.Dispose();
objSqlCon.ResetStatistics();
slCons.RemoveAt(nIndex);
}
}
}
internal class UserA
{
public string ConnectionID { get { return HttpContext.Current.Session.SessionID; } }
private ConnA objConnA = new objConnA();
public void ConnectDB()
{
objConnA.CreateCon(ConnectionID));
}
public void DisConnectDB()
{
objConnA.CloseConnection(ConnectionID));
}
}
Access to the SortedList isn't thread safe.
In CreateCon, two threads could access this simultaneously:
if (connSuccess && slCons.ContainsKey(ConnID) == false)
Both threads could determine that the key isn't present, and then both threads try to add it, so that one of them fails.
In method 2:
When this is called - slCons.RemoveAt(nIndex); - the lock guarantees that another call to the same method won't remove another connection, which is good. But nothing guarantees that another thread won't call CreateCon and insert a new connection string, changing the indexes so that nIndex now refers to a different item in the collection. You would end up closing, disposing, and deleting the wrong connection string, likely one that another thread was still using.
It looks like you're attempting an orchestration which ensures that a single connection string will be used across multiple operations. But there's no need to introduce that complication. Whatever class or method needs a connection, there's no need for it to collaborate with this collection and these methods. You can just let each of them open a connection when it needs it and dispose the connection when it's done.
That's expensive, but that's why the framework implements connection pooling. From the perspective of your code connections are being created, opened, closed, and disposed.
But behind the scenes, the "closed" connection isn't really closed, at least not right away. It's actually kept open. If, in a brief period, you "open" another connection with the same connection string, you're actually getting the same connection again, which is still open. That's how the number of connections opened and closed is reduced without us having to manually manage it.
That in turn prevents us from having to do what it looks like you're doing. This might be different if we were opening a transaction on a connection, and then we had to ensure that multiple operations were performed on the same connection. But even then it would likely be clearer and simpler to pass around the connection, not an ID.
I am testing out ServiceStacks OrmLite. I have previosly used MySql without OrmLite and now I am faced with the problem easiest described in this error message:
There is already an open DataReader associated with this Connection
which must be closed first.
Since I have a multi-threaded application, certain threads will be polling the database, while other will insert, update or select "on demand", when needed. This results in the above mentioned exception.
What I need to do is to be able to detect if a connection (IDbHandler) is "busy"; has an open DataReader or something else that. If it is busy, take the next connection (from the "connection pool" i want to implement). The problem is, there is no method or property I can use in the IDbHandler object to determine if it is busy or not.
I have solved this in the "normal" mysql case by simply having a method where I send in the MySqlCommand or just the query string, like:
dbConnections.ExecuteQuery("SELECT * FROM test");
dbConnections.ExecuteQuery(cmd); // cmd == MySqlCommand
and the ExecuteQuery will handle of finding an open connection and just passing on the cmd/query there.
But, since I am using OrmLite it has a lot of extension methods to IDbConnection and I do not want to create "proxy methods" for each one. In the simple mysql case above, there is really only one method needed, that takes in a MySqlCommand, but not so with the many methods in OrmLite.
The first question:
How can I detect if a connection is busy? I want to avoid a try-catch situation to detect it.
Second question:
Is there some way to pass the entire "method" call, something like:
Example:
dbConnections.Run(iDbHandler.Select<MyObject>(q => q.Id > 10));
// or
dbConnections.Run(iDbHandler.Where<MyObject>(q => q.Id > 10));
// or
dbConnections.Run(iDbHandler.SomeOtherWeirdMetod<MyObject>(q => bla bla bla));
This is by far not the best solution, but it is an approach that I am testing with to see how it handles for my specific case (currently on ver 3.97). Like Ted, I am seeing frequent exceptions of open data readers, or connections being returned as closed.
In my usage all services inherit my parent service (which in turn inherits Service) that handles some common meta-data handling. I have opted to have my base service override the Service's Db property and do a quick check on the Db connection's state and attempt a recovery. One case this fixed immediately was the following:
My MsSql server is running in a failover cluster. When the SQL server flips from node A to node B, there is no built-in mechanism that I found in ServiceStack to detect that its in memory connections are "dirty" and need to reconnect.
Comments and improvements are most welcome.
public override System.Data.IDbConnection Db
{
get
{
try
{
var d = base.Db;
if (d == null || d.State != System.Data.ConnectionState.Open)
return ForceNewDbConn();
else
return d;
}
catch (Exception ex)
{
return ForceNewDbConn();
//throw;
}
}
}
private IDbConnection ForceNewDbConn()
{
try
{
var f = TryResolve<IDbConnectionFactory>();
if (f as OrmLiteConnectionFactory != null)
{
var fac = f as OrmLiteConnectionFactory;
fac.AutoDisposeConnection = true;
var newDBconn = fac.Open();
return newDBconn;
}
return base.Db;
}
catch (Exception ex)
{
throw;
}
}
I have a Linq-to-SQL DataClasses object that I utilize to make database calls. I've wrapped it like so:
public class DataWrapper {
private DataClassesDataContext _connection = null;
private static DataWrapper _instance = null;
private const string PROD_CONN_STR = "Data Source=proddb;Initial Catalog=AppName;User ID=sa;Password=pass; MultipleActiveResultSets=true;Asynchronous Processing=True";
public static DataClassesDataContext Connection {
get {
if (Instance._connection == null)
Instance._connection = new DataClassesDataContext(DEV_CONN_STR);
return Instance._connection;
}
}
private static DataWrapper Instance {
get {
if (_instance == null) {
_instance = new DataWrapper();
}
return _instance;
}
}
}
I have a couple of threads using this wrapper to make stored procedure calls, like this:
DataWrapper.Connection.Remove_Message(completeMessage.ID);
On very rare occasions, my DataClasses object will throw the exception:
ExecuteNonQuery requires an open and available Connection. The
connection's current state is closed.
I'm not managing the connection's state in any way -- I figured Linq-to-SQL should handle this. I could check the connection state of the Connection each time I make a call and open it if it has been closed but that seems like a hack.
I've tried putting MultipleActiveResultSets=true and Asynchronous Processing=True on the connection string to try to handle the possibility of SQL forcibly closing connections, but that hasn't seemed to help.
Any ideas?
You should not cache and re-use a DB connection object ... especially from multiple threads.
You should open a connection, execute your operation(s) and close your connection each time you need to access the DB.
The underlying database access infrastructure (ASP.NET/OLEDB) will manage connection pooling in such a way that reduces most re-connection costs to (effectively) zero.
I have a class Worker which sends emails periodically,I start in Global.asax.cs on App_start()
public static class Worker
{
public static void Start()
{
ThreadPool.QueueUserWorkItem(o => Work());
}
public static void Work()
{
var r = new DbContext();
var m = new MailSender(new SmtpServerConfig());
while (true)
{
Thread.Sleep(600000);
try
{
var d = DateTime.Now.AddMinutes(-10);
var ns = r.Set<Notification>().Where(o => o.SendEmail && !o.IsRead && o.Date < d);
foreach (var n in ns)
{
m.SendEmailAsync("noreply#example.com", n.Email, NotifyMailTitle(n) + " - forums", NotifyMailBody(n));
n.SendEmail = false;
}
r.SaveChanges();
}
catch (Exception ex)
{
ex.Raize();
}
}
}
}
So I keep this dbcontext alive for the entire lifetime of the application is this a good practice ?
DbContext is a very light-weight object.
It doesn't matter whether your DbContext stays alive or you instantiate it just before making the call because the actual DB Connection only opens when you SubmitChanges or Enumerate the query (in that case it is closed on end of enumeration).
In your specific case. It doesn't matter at all.
Read Linq DataContext and Dispose for details on this.
I would wrap it in a using statement inside of Work and let the database connection pool do it's thing:
using (DbContext r = new DbContext())
{
//working
}
NOTE: I am not 100% sure how DbContext handles the db connections, I am assuming it opens one.
It is not good practice to keep a database connection 'alive' for the lifetime of an application. You should use a connection when needed and close it via the API(using statement will take care of that for you). The database connection pool will actually open and close connections based on connection demands.
I agree with #rick schott that you should instantiate the DbContext when you need to use it rather than keep it around for the lifetime of the application. For more information, see Working with Objects (Entity Framework 4.1), especially the section on Lifetime:
When working with long-running context consider the following:
As you load more objects and their references into memory, the
memory consumption of the context may increase rapidly. This may cause
performance issues.
If an exception causes the context to be in an unrecoverable state,
the whole application may terminate.