Develop Reference

In-memory SQLite and database disappearing

I am using Nancy to create an api to a database, and am wanting to test / develop against an in-memory database.
I am using an instance of my database class in a custom bootstrapper class with the connection string Data Source=:memory:, which in turn is creating the necessary tables - I have stepped through this and I'm confident this is occuring.
I am then obtaining a new connection using the same connection string to load/save data, but even a simple select is coming up with the sql error that the table doesn't exist.
Is there a fault in this logic of creating and using a new connection with the same connection string?

OK, so straight from the docs:
The database ceases to exist as soon as the database connection is closed. "
However, this can be worked around with multiple connections by using cache=shared in your connection string.
However, this isn't a solution to the problem because as soon as the last connection closes, the database ceases to exist.

You need to keep the SQLiteConnection open and ensure that the DbContext does not own the connection. This way, when the DbContext is disposed by the container, the connection doesn't get closed with it. Note: This only works with EF6. You can still pass the flag in EF5, but the context will still close the connection when the context gets disposed.
I created a TestBootstrapper which inherited the working Bootstrapper from the web project.
As part of the ConfigureRequestContainer method, I used a method factory on the DbContext registration which created a new DbContext each time, but used the existing connection. If you don't do this, the DbContext will get disposed after your first request and the second request will fail.
public class TestBootstrapper : Bootstrapper
{
private const string ConnectionString = "data source=:memory:;cache=shared;";
private static SQLiteConnection _connection;
private static TestInitializer _initializer = new TestInitializer();
protected override void ConfigureRequestContainer(TinyIoCContainer, NancyContext context)
{
container.Register<Context>((_,__) =>
{
if (_connection == null)
_connection = new SQLiteConnection(ConnectionString);
// The false flag tells the context it does not own the connection, i.e. it cannot close it. (EF6 behaviour only)
var dbContext = new Context(_connection, _initializer, false);
if (_connection.State == ConnectionState.Closed)
_connection.Open();
// I build the DB and seed it here
_initializer.InitializeDatabase(context);
return dbContext;
});
// Additional type registrations
}
// Call this method on a [TearDown]
public static void Cleanup()
{
if (_connection != null && _connection.State == ConnectionState.Open)
_connection.Close();
_connection = null;
}
}

When will connection be closed in case of IEnumerable with using

Suppose I have such pseudo code using some pseudo ORM (ok in my case it's Linq2Db).
static IEnumerable<A> GetA()
{
using (var db = ConnectionFactory.Current.GetDBConnection())
{
return from a in db.A
select a;
}
}
static B[] DoSmth()
{
var aItems = GetA();
if (!aItems.Any())
return null;
return aItems.Select(a => new B(a.prop1)).ToArray();
}
When will Connection be closed in db? Would it be closed at all in that case? What connection would be closed - those in using statement or those in lambda expression? .NET compiler is creating anonymous class for lambdas, so it will copy connection to that class. When would that connection be closed?
Somehow I managed to get Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. I materialized queries and exception disappeared. But I'm wondering how this thing works.

This is entirely dependent on your ORM. If disposing of ConnectionFactory.Current.GetDBConnection() closes the connection then you will never be able to enumerate the result. If it doesnt close the connection (and something else does) it might work depending on if someone else has closed the connection.
In any case you probably dont want to return an un-enumerated enumerable from something which creates and disposes the connection.
either enumerate the collection before closing it eg:
static IEnumerable<A> GetA()
{
using (var db = ConnectionFactory.Current.GetDBConnection())
{
return (from a in db.A
select a).ToArray();
}
}
or control the connection at the level which does enumerate the results eg:
static IEnumerable<A> GetA(whatevertype db)
{
return from a in db.A
select a;
}
static B[] DoSmth()
{
using (var db = ConnectionFactory.Current.GetDBConnection())
{
var aItems = GetA(db);
if (!aItems.Any())
return null;
return aItems.Select(a => new B(a.prop1)).ToArray();
}
}

Linq2db release connection when you close your connection object. In your example it happens when you leave using block and db object is disposed. This means that you cannot return query from connection scope - you need to fetch your data first or postpone connection disposal till you didn't get all required data.
Exception you observed was due to error in linq2db prior to 1.8.0 where it allowed query execution on disposed connection, which lead to connection leak. See https://github.com/linq2db/linq2db/issues/445 for more details. After that fix you cannot write code like GetA() in your example as you will get ObjectDisposedException when you will try to enumerate results.

after first call :
in this call connection opened in using scope , without any fetching data at the end of usnig scope Closed.
var aItems = GetA();
but at this line :
if (!aItems.Any())
there isn't any open connection

Database locked issue while Inserting in same table the Array of more than 1000 records by multiple client

I am facing the big issue. I have created the service stack web services in C# DotNet and Database is SQLite (v4.0.30319). My System hang / Database locked issue, some time SQlite database file also corrupted while inserting/ updating the database table by number of clients (more than one connection). All are update the same table same time. Whle One is updating other try to connect than the system hang. I have tried all the ways but not able to resolve this issue.
How can I force that One One client can update the table at one time other clients can readonly mode.
My code as below
public bool CreateAnother2(LTHR record)
{
public IDbConnection Db { get; private set; }
OrmLiteConnectionFactory factory = new OrmLiteConnectionFactory("Data Source= C:/Database/db.sqlite;Integrated Security=True;User Instance=True", ServiceStack.OrmLite.Sqlite.SqliteOrmLiteDialectProvider.Instance);
try
{
using (Db = factory.OpenDbConnection())
{
using (var trans = Db.BeginTransaction()
{
// If no error insert data
var id = (int)Db.Insert(record, selectIdentity: true);
trans.Commit();
Db.Close();
return true;
}
}
}
catch (Exception ex)
{
if (Db.State == ConnectionState.Open)
{
Db.Close();
}
}
finally
{
if (Db.State == ConnectionState.Open)
{
Db.Close();
}
}
}

Sqlite is not meant to be used concurrently by multiple clients. That is your application runnning on multiple machines, or multiple applications on the same machine accessing the same shared database-file.
There are other solutions for that use case. (think mysql, postgress, Sqlserver, Oracle ea.).
If your application uses multiple connections to the database from one application but on different threads make sure that you open the database in serialized mode.
https://www.sqlite.org/threadsafe.html

Should I keep an instance of DbContext in a separate thread that performs periodic job

I have a class Worker which sends emails periodically,I start in Global.asax.cs on App_start()
public static class Worker
{
public static void Start()
{
ThreadPool.QueueUserWorkItem(o => Work());
}
public static void Work()
{
var r = new DbContext();
var m = new MailSender(new SmtpServerConfig());
while (true)
{
Thread.Sleep(600000);
try
{
var d = DateTime.Now.AddMinutes(-10);
var ns = r.Set<Notification>().Where(o => o.SendEmail && !o.IsRead && o.Date < d);
foreach (var n in ns)
{
m.SendEmailAsync("noreply#example.com", n.Email, NotifyMailTitle(n) + " - forums", NotifyMailBody(n));
n.SendEmail = false;
}
r.SaveChanges();
}
catch (Exception ex)
{
ex.Raize();
}
}
}
}
So I keep this dbcontext alive for the entire lifetime of the application is this a good practice ?

DbContext is a very light-weight object.
It doesn't matter whether your DbContext stays alive or you instantiate it just before making the call because the actual DB Connection only opens when you SubmitChanges or Enumerate the query (in that case it is closed on end of enumeration).
In your specific case. It doesn't matter at all.
Read Linq DataContext and Dispose for details on this.

I would wrap it in a using statement inside of Work and let the database connection pool do it's thing:
using (DbContext r = new DbContext())
{
//working
}
NOTE: I am not 100% sure how DbContext handles the db connections, I am assuming it opens one.
It is not good practice to keep a database connection 'alive' for the lifetime of an application. You should use a connection when needed and close it via the API(using statement will take care of that for you). The database connection pool will actually open and close connections based on connection demands.

I agree with #rick schott that you should instantiate the DbContext when you need to use it rather than keep it around for the lifetime of the application. For more information, see Working with Objects (Entity Framework 4.1), especially the section on Lifetime:
When working with long-running context consider the following:
As you load more objects and their references into memory, the
memory consumption of the context may increase rapidly. This may cause
performance issues.
If an exception causes the context to be in an unrecoverable state,
the whole application may terminate.

DBTransactions between stateless calls using GUIDs

I'm looking to add transactional support to my DB engine and providing to Abstract Transaction Handling down to passing in Guids with the DB Action Command. The DB engine would run similar to:
private static Database DB;
public static Dictionary<Guid,DBTransaction> Transactions = new ...()
public static void DoDBAction(string cmdstring,List<Parameter> parameters,Guid TransactionGuid)
{
DBCommand cmd = BuildCommand(cmdstring,parameters);
if(Transactions.ContainsKey(TransactionGuid))
cmd.Transaction = Transactions[TransactionGuid];
DB.ExecuteScalar(cmd);
}
public static BuildCommand(string cmd, List<Parameter> parameters)
{
// Create DB command from EntLib Database and assign parameters
}
public static Guid BeginTransaction()
{
// creates new Transaction adding it to "Transactions" and opens a new connection
}
public static Guid Commit(Guid g)
{
// Commits Transaction and removes it from "Transactions" and closes connection
}
public static Guid Rollback(Guid g)
{
// Rolls back Transaction and removes it from "Transactions" and closes connection
}
The Calling system would run similar to:
Guid g
try
{
g = DBEngine.BeginTransaction()
DBEngine.DoDBAction(cmdstring1, parameters,g)
// do some other stuff
DBEngine.DoDBAction(cmdstring2, parameters2,g)
// sit here and wait for a response from other item
DBEngine.DoDBAction(cmdstring3, parameters3,g)
DBEngine.Commit(g)
}
catch(Exception){ DBEngine.Rollback(g);}
Does this interfere with .NET connection pooling (other than a connection be accidently left open)?
Will EntLib keep the connection open until the commit or rollback?

The connection will be kept open until a commit or rollback. It is the transaction that is keeping the connection open.
It will not affect connection pooling, other than a connection held by a transaction will not be returned to the connection pool.
I would recommend that you look at the .net TransactionScope. This may be able to meet your needs, without you writing any of this custom code.