Separating concerns for NpgsqlConnection - c#

I have an architecture where multiple repository classes implement different queries and commands that are run against the database. I would like to separate the concern of "connecting to the database" + "running queries" and "providing a query to run" + "treating the result". I've written a Connection class that is then passed as a constructor argument to the repositories like such:
public class PostgreSqlConnection
{
private string connectionString;
public PostgreSqlConnection(string connectionString)
{
this.connectionString = connectionString;
}
public async Task<NpgsqlDataReader> ExecuteQueryCommand(NpgsqlCommand command)
{
using NpgsqlConnection connection = new NpgsqlConnection(this.connectionString);
await connection.OpenAsync();
command.Connection = connection;
command.Prepare();
return await command.ExecuteReaderAsync();
}
public async Task ExecuteNonQueryCommand(NpgsqlCommand command)
{
using NpgsqlConnection connection = new NpgsqlConnection(this.connectionString);
await connection.OpenAsync();
command.Connection = connection;
command.Prepare();
await command.ExecuteNonQueryAsync();
}
}
The instantiation would look something like this:
PostgreSqlConnection connection = new PostgreSqlConnection("...connection string");
IRepositoryA repA = new PostgreSqlRepositoryA(connection);
IRepositoryB repB = new PostgreSqlRepositoryB(connection);
Code duplication aside, this doesn't work since in the query case, the connection would be disposed of at the end of the ExecuteQueryCommand method and the reader would stop working.
Removing the using statement would fix this but from what I gather that is not good practice. Writing a Dispose / Disconnect method that I could call in the repositories would be something that would also work but it's not the repository's job to dispose of the connection.
How could I go about keeping the concerns separated and disposing of the items properly?

I think what here can help is a Unit of Work pattern. Basically, with UnitOfWork class you handle the connection, repository instances and transactions if needed (in case you are saving data to DB). You also have a flexibility to open a connection/transaction and then execute multiple commands across many repositories and in the end you either commit or rollback the transaction, or in case of pure reading you just close the connection using IDisposable pattern.
UnitOfWork class will handle the connection part, your BaseRepository (abstract class) will have your generic execute methods and it will handle the execution of query/command. The concrete repositories (A and B in your case) will inherit from BaseRepository, just prepare the commands/queries and call the Execute methods from a BaseRepository. Their responsibility is basically to prepare the query/command and to handle the result from Execute methods.
Please review the code because I don't have Postgres database and I can't test it 100%. I hope this is enough for you to give you a direction and the main idea behind the approach.
Here is the idea:
Implement UnitOfWork where you manange the connection, transaction if needed and repository instances.
public class UnitOfWork : IDisposable
{
private PostgreSqlRepositoryA _postgreSqlRepositoryA;
private PostgreSqlRepositoryB _postgreSqlRepositoryB;
private NpgsqlConnection _sqlConnection;
private NpgsqlTransaction _sqlTransaction;
private bool _disposed;
public UnitOfWork(string connectionString, bool withTransaction)
{
_sqlConnection = new NpgsqlConnection(connectionString);
_sqlConnection.Open();
if (withTransaction)
_sqlTransaction = _sqlConnection.BeginTransaction();
}
public PostgreSqlRepositoryA PostgreSqlRepositoryA
{
get
{
if(_postgreSqlRepositoryA == null)
{
_postgreSqlRepositoryA = new PostgreSqlRepositoryA(_sqlConnection);
}
return _postgreSqlRepositoryA;
}
}
public PostgreSqlRepositoryB PostgreSqlRepositoryB
{
get
{
if (_postgreSqlRepositoryB == null)
{
_postgreSqlRepositoryB = new PostgreSqlRepositoryB(_sqlConnection);
}
return _postgreSqlRepositoryB;
}
}
public void Commit()
{
// hanlde using try-catch
if(_sqlTransaction != null)
{
_sqlTransaction.Commit();
}
}
public void Rollback()
{
if (_sqlTransaction != null)
{
_sqlTransaction.Rollback();
}
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (!this._disposed)
{
if (_sqlTransaction != null)
{
_sqlTransaction.Rollback(); // or throw an Exception for an opened transaction
}
if (_sqlConnection != null)
{
_sqlConnection.Close();
_sqlConnection.Dispose();
}
this._disposed = true;
}
}
}
Then what you need is a BaseRepository which will hold your "generic" methods for query/command execution:
public abstract class BaseRepository
{
protected NpgsqlConnection _sqlConnection;
public BaseRepository(NpgsqlConnection sqlConnection)
{
this._sqlConnection = sqlConnection;
}
public async Task<NpgsqlDataReader> ExecuteQueryCommand(NpgsqlCommand command)
{
command.Connection = _sqlConnection;
command.Prepare();
return await command.ExecuteReaderAsync();
}
public async Task ExecuteNonQueryCommand(NpgsqlCommand command)
{
command.Connection = _sqlConnection;
command.Prepare();
await command.ExecuteNonQueryAsync();
}
}
Your concrete repository implementation will look like this (I didn't bother with handling the reader, you can add that part):
public class PostgreSqlRepositoryB : BaseRepository
{
public PostgreSqlRepositoryB(NpgsqlConnection sqlConnection)
: base(sqlConnection)
{}
public async Task<int> GetCountB()
{
using (var sqlCommand = new NpgsqlCommand())
{
sqlCommand.CommandText = "select count(1) from TableB";
var reader = await ExecuteQueryCommand(sqlCommand);
// TODO: handle reader
}
}
}
And the in the end you will use it from your service or client method like this (if your are just reading from DB then set withTransaction to false, you don't need a transaction in that case):
using(var uow = new UnitOfWork("place_your_conn_string", withTransaction: true))
{
var countB = await uow.PostgreSqlRepositoryB.GetCountB();
uow.PostgreSqlRepositoryB.SaveSomethingToA("123456");
uow.Commit();
}

Related

Using interface and abstract class in repository pattern in ASP.NET Core

We are currently using dapper ORM to access data by calling store procedures. The current code is having a class BusinessFunctions which inherits another class DataFunctions which are having helper methods to execute the stored procedures.
I am not happy with this code. It's just too rigid and not future proof. And above all it's not coded to an interface rather coded to an implementation. I propose an interface IRepository with an abstract class Repository which implements all helper generic methods. Then I create BusinessRepository that implements the abstract Repository class and call the generic helpers method. Again, my colleague is telling me to remove the IRepository interface and just use the Repository abstract class.
public class BusinessFunctions : DataFunctions
{
public BusinessFunctions(ConnectionManager conMgr, LogWriter logWriter, AppUser appUser) : base(conMgr, logWriter, appUser)
{
}
public async Task<Business> FindAsync(int businessId)
{
throw new NotImplementedException();
}
public async Task<Business> FindAsync(string businessGuid)
{
var lst = await StoredProcQueryAsync<Business>("spBusinessGetSetupGUID", new { BusinessGuid = businessGuid });
if (lst.Count() == 0)
throw new NotFoundInDatabaseException("Business", businessGuid);
else
return lst.Single();
}
public async Task<bool> IsHostedTokenizeCardAllowedAsync(string businessGuid)
{
var b = await FindAsync(businessGuid);
if (b.HostedPaymentEnabled)
return true;
else
return false;
}
}
public class DataFunctions : IDisposable
{
private ConnectionManager _conMgr;
private LogWriter _logWriter;
private AppUser _appUser;
public ConnectionManager ConnMgr
{
get { return _conMgr; }
}
protected LogWriter Logger
{
get { return _logWriter; }
}
protected AppUser User
{
get { return _appUser; }
}
public DataFunctions(ConnectionManager conMgr, LogWriter logWriter, AppUser appUser)
{
_conMgr = conMgr;
_logWriter = logWriter;
_appUser = appUser;
}
public void Dispose()
{
}
public async Task StoredProcExecuteNonQueryAsync(string storedProc,
List<StoredProcParameter> storedProcParameters = null,
SqlCommandTimeout commandTimeout = SqlCommandTimeout.Default,
SqlAccessType accessType = SqlAccessType.ReadWrite
)
{
using (SqlConnection conn = new SqlConnection(ConnMgr.SqlConnectionString))
{
await conn.OpenAsync();
await StoredProcExecuteNonQueryAsync(conn,
storedProc,
storedProcParameters: storedProcParameters,
commandTimeout: commandTimeout,
accessType: accessType);
}
}
public async Task StoredProcExecuteNonQueryAsync(SqlConnection conn,
string storedProc,
List<StoredProcParameter> storedProcParameters = null,
SqlCommandTimeout commandTimeout = SqlCommandTimeout.Default,
SqlAccessType accessType = SqlAccessType.ReadWrite,
SqlTransaction trans = null
)
{
using (SqlCommand cmd = new SqlCommand(storedProc, conn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandTimeout = (int)commandTimeout;
if (trans != null) cmd.Transaction = trans;
if (storedProcParameters != null)
{
foreach(var p in storedProcParameters)
{
cmd.Parameters.Add(p.ToSqlParameter());
}
}
await cmd.ExecuteNonQueryAsync();
}
}
public async Task<IEnumerable<T>> StoredProcQueryAsync<T>(string storedProc,
object storedProcParameters = null,
SqlCommandTimeout commandTimeout = SqlCommandTimeout.Default,
SqlAccessType accessType = SqlAccessType.ReadWrite)
{
using (SqlConnection conn = new SqlConnection(ConnMgr.SqlConnectionString))
{
conn.Open();
return await StoredProcQueryAsync<T>(conn,
storedProc,
storedProcParameters,
commandTimeout);
}
}
public async Task<IEnumerable<T>> StoredProcQueryAsync<T>(SqlConnection conn,
string storedProc,
object storedProcParameters = null,
SqlCommandTimeout commandTimeout = SqlCommandTimeout.Default)
{
return await conn.QueryAsync<T>(storedProc,
commandType: CommandType.StoredProcedure,
commandTimeout: (int)commandTimeout,
param: storedProcParameters);
}
}
I think the reason you're unhappy with the code is that it seems to be intermingling service functionality into the repository layer. The repository layer should simply call the stored procedure.
public async Task<bool> IsHostedTokenizeCardAllowedAsync(string businessGuid)
{
var b = await FindAsync(businessGuid);
if (b.HostedPaymentEnabled)
return true;
else
return false;
}
This for example is a good candidate to be in the service layer.
Your repo layer should really just have your ConnectionManager or a Connection factory injected via IoC.
The trick we use is to put an attribute on data model fields that we know are going to be stored procedure parameters (usually most or all). Then we have an extension method that reflects over the attributes and pulls the fields, values, and types creating a dapper DynamicParameters object. Most of our repository calls look like this:
public async Task<User> AddUserAsync(UserAdd user)
{
using (var connection = _connectionFactory.Create()
{
var result = connection.ExecuteAsync("dbo.AddUser", user.GetParameters(), commandType: CommandType.StoredProcedure";
return result;
}
}
It's relatively quick and easy to use. Gets are very easy to test. Inserts/Deletes/Updates not so much. You get into needing to mock the SqlConnection which can be problematic.
In addition, if you get into more complex areas that are subject to change, you can use the strategy pattern to move methods into their own classes. Below would be an example of splitting your add method into its own class:
public class MyRepository
{
private readonly IAddMethod<UserAdd> _addMethod;
private readonly IConnectionFactory _connectionFactory;
public MyRepository(IAddMethod<UserAdd> userAddMethod,
IConnectionFactory connectionFactory)
{
//..guard clauses, assignments, etc.
}
public async Task<int> AddAsync(UserAdd user)
{
return _addMethod.AddAsync(user);
}
}
You can even decorate these strategy methods in IoC to hide/augment them as needed. (in structuremap it's .DecorateAllWith()
In short, move any logic to the service layer, consider a generic extension method for creating your DynamicParameters list, and inject the connection factory via IoC. I think you'll find the separation of concerns will simplify things significantly.

How to implement the Unit of Work(UoW) pattern with ADO.NET and use it in ASP.NET MVC

I know this question has been asked before, but in those questions there aren't enough details about the actual implementation, so I decided to search for some information about how can this be achieve and this is what I've got so far:
The IUnitOfWork Interface:
public interface IUnitOfWork : IDisposable
{
IDomainTableRepository DomainTables { get; }
IVariableRepository Variables { get; }
IModelRepository Models { get; }
IStructureRepository Structures { get; }
ISentenceRepository Sentences { get; }
IExpressionRepository Expressions { get; }
IReturnRepository Returns { get; }
void Commit();
}
The implementation for IUnitOfWork:
public class SqlUnitOfWork : IUnitOfWork
{
const string ConnectionStringName = "DefaultConnection";
private AdoNetContext _context;
private DomainTableRepository _domainTables;
private VariableRepository _variables;
private ModelRepository _models;
private StructureRepository _structures;
private SentenceRepository _sentences;
private ExpressionRepository _expressions;
private ReturnRepository _returns;
public SqlUnitOfWork()
{
var connectionString =
ConfigurationManager
.ConnectionStrings[ConnectionStringName]
.ConnectionString;
_context = new AdoNetContext(connectionString, true);
}
public IDomainTableRepository DomainTables
{
get
{
if (_domainTables == null)
{
_domainTables = new DomainTableRepository(_context);
}
return _domainTables;
}
}
//...getters for the remaining repositories
public void Commit()
{
_context.SaveChanges();
}
#region IDisposable Support
private bool disposedValue = false;
protected virtual void Dispose(bool disposing)
{
if (!disposedValue)
{
if (disposing)
{
_context.Dispose();
}
disposedValue = true;
}
}
public void Dispose()
{
Dispose(true);
}
#endregion
}
The AdoNetContext class
All the credit for the implementation of this class goes to #jgauffin.
His original blog post can be found here.
public class AdoNetContext : IDisposable
{
private IDbConnection _connection;
private bool _ownsConnection;
private IDbTransaction _transaction;
public AdoNetContext(string connectionString, bool ownsConnection)
{
_connection = new SqlConnection(connectionString);
_connection.Open();
_ownsConnection = ownsConnection;
_transaction = _connection.BeginTransaction();
}
public IDbCommand CreateCommand()
{
var command = _connection.CreateCommand();
command.Transaction = _transaction;
return command;
}
public void SaveChanges()
{
if (_transaction == null)
{
throw new InvalidOperationException("Transaction have already been already been commited. Check your transaction handling.");
}
_transaction.Commit();
_transaction = null;
}
public void Dispose()
{
if (_transaction != null)
{
_transaction.Rollback();
_transaction = null;
}
if (_connection != null && _ownsConnection)
{
_connection.Close();
_connection = null;
}
}
}
The implementation of a repository(DomainTableRepository) that uses AdoNetContext:
public class DomainTableRepository : IDomainTableRepository
{
private AdoNetContext _context;
public DomainTableRepository(AdoNetContext context)
{
_context = context;
}
public DomainTable Get(int id)
{
DomainTable table;
using (var commandTable = _context.CreateCommand())
{
commandTable.CommandType = CommandType.StoredProcedure;
commandTable.CommandText = "up_DomainTable_GetById";
commandTable.Parameters.Add(commandTable.CreateParameter("#pId", id));
table = ToList(commandTable).FirstOrDefault();
}
return table;
}
public IEnumerable<DomainTable> GetAll(int pageIndex = 1, int pageSize = 10)
{
using (var command = _context.CreateCommand())
{
command.CommandType = CommandType.StoredProcedure;
command.CommandText = "up_DomainTable_GetAll";
command.Parameters.Add(command.CreateParameter("#pPageIndex", pageIndex));
command.Parameters.Add(command.CreateParameter("#pPageSize", pageSize));
return ToList(command).ToList();
}
}
public int Remove(DomainTable entity)
{
using (var command = _context.CreateCommand())
{
command.CommandType = CommandType.StoredProcedure;
command.CommandText = "up_DomainTable_Delete";
command.Parameters.Add(command.CreateParameter("#pId", entity.Id));
return command.ExecuteNonQuery();
}
}
}
I apologize for posting so much code,but I want to make sure you understand everything about what I want to ask.
First, I created an interface for the UoW(IUnitOfWork), because in the future I might want to change to an ORM like EF, but since I'm just taking my first step into the the world of back-end programming I wanted to start with ADO.NET and stored procedures.
So here are my questions regarding the implementation of the UoW pattern with ADO.NET and its usage in ASP.NET MVC:
1)Given the code for the AdoNetContext(the one that manages the db connection and transactions), is there any performance penalty for always creating a transaction?
This is what I read in a SQL book :"Executing a SELECT
statement within a transaction can create locks on the referenced tables, which can in turn block other users or sessions from performing work or reading data" Do the transactions created in C# behave exactly as their counterparts in SQL?
2)As you can see in IUnitOfWork and its implementation (UnitOfWork), I have readonly properties for every repository I have in my application. This a common "pattern" I've seen in lots of tutorials on EF and I found it very convenient since every time I new up a instance of IUnitOfWork I already have access to all its repositories and I don't have to clutter my code with the instantiation of the repositories I need in a specific Action (from a Controller).
Do you think it would too much overhead to instantiate all the repositories (as SQLUnitOfWork does) every time I create/inject a new instance of IUnitOfWork? There are some scenarios in which I might work with 4 or 5 repositories at the same time, but in most cases I'll be using just 1 or 2 in the same action.
3) I would like to use DI (maybe Ninject) to inject an instance of IUnitOfWork into my Controllers. This way later when I use another persistence framework, I will only have to change this line kernel.Bind<IUnitOfWork>().To<SqlUnitOfWork>(); From SqlUnitOfWork, to let's say EFUnitOfWork.
The problem I've encountered here is that the Dispose methods in SqlUnitOfWork and AdoNetContext are never called and as much as I want to use DI I'd rather wrap the instantiation of SqlUnitOfWork in an using statement in every Action instead of be leaking memory just so that I can use DI.
So this is what I wanted to ask you guys.
I'm trying to learn so please before clicking the close link or marking the question as a duplicate take some time to read my question.
Thanks.

C#: How to Wrap my Unit of Work code in my repository classes

See my repository code
public abstract class AdoRepository<T> where T : class
{
private static SqlConnection _connection;
public AdoRepository(string connectionString)
{
_connection = new SqlConnection(connectionString);
}
public virtual T PopulateRecord(SqlDataReader reader)
{
return null;
}
public virtual void GetDataCount(int count)
{
}
protected IEnumerable<T> GetRecords(SqlCommand command)
{
var list = new List<T>();
command.Connection = _connection;
_connection.Open();
try
{
var reader = command.ExecuteReader();
try
{
while (reader.Read())
{
list.Add(PopulateRecord(reader));
}
reader.NextResult();
if (reader.HasRows)
{
while (reader.Read())
{
GetDataCount(Convert.ToInt32(reader["Count"].ToString()));
}
}
}
finally
{
// Always call Close when done reading.
reader.Close();
}
}
catch(Exception ex)
{
string Msg = ex.Message;
}
finally
{
_connection.Close();
}
return list;
}
protected T GetRecord(SqlCommand command)
{
T record = null;
command.Connection = _connection;
_connection.Open();
try
{
var reader = command.ExecuteReader();
try
{
while (reader.Read())
{
record = PopulateRecord(reader);
break;
}
}
finally
{
// Always call Close when done reading.
reader.Close();
}
}
finally
{
_connection.Close();
}
return record;
}
protected IEnumerable<T> ExecuteStoredProc(SqlCommand command)
{
var list = new List<T>();
command.Connection = _connection;
command.CommandType = CommandType.StoredProcedure;
_connection.Open();
try
{
var reader = command.ExecuteReader();
try
{
while (reader.Read())
{
var record = PopulateRecord(reader);
if (record != null) list.Add(record);
}
}
finally
{
// Always call Close when done reading.
reader.Close();
}
}
finally
{
_connection.Close();
}
return list;
}
}
public class StudentRepository : AdoRepository<Student>
{
public int DataCounter { get; set; }
public StudentRepository(string connectionString)
: base(connectionString)
{
}
public IEnumerable<Student> GetAll()
{
// DBAs across the country are having strokes
// over this next command!
using (var command = new SqlCommand("SELECT ID, FirstName,LastName,IsActive,StateName,CityName FROM vwListStudents"))
{
return GetRecords(command);
}
}
public Student GetById(string id)
{
// PARAMETERIZED QUERIES!
using (var command = new SqlCommand("SELECT ID, FirstName,LastName,IsActive,StateName,CityName FROM vwListStudents WHERE Id = #id"))
{
command.Parameters.Add(new ObjectParameter("id", id));
return GetRecord(command);
}
}
public IEnumerable<Student> GetStudents(int StartIndex, int EndIndex, string sortCol, string sortOrder)
{
string strSQL = "SELECT * FROM vwListStudents WHERE ID >=" + StartIndex + " AND ID <=" + EndIndex;
strSQL += " ORDER BY " + sortCol + " " + sortOrder;
strSQL += ";SELECT COUNT(*) AS Count FROM vwListStudents";
var command = new SqlCommand(strSQL);
return GetRecords(command);
}
public override Student PopulateRecord(SqlDataReader reader)
{
return new Student
{
ID = Convert.ToInt32(reader["ID"].ToString()),
FirstName = reader["FirstName"].ToString(),
LastName = reader["LastName"].ToString(),
IsActive = Convert.ToBoolean(reader["IsActive"]),
StateName = reader["StateName"].ToString(),
CityName = reader["CityName"].ToString()
};
}
public override void GetDataCount(int count)
{
DataCounter = count;
}
}
this way unit of work code added
public class UnitOfWorkFactory
{
public static IUnitOfWork Create()
{
var connection = new SqlConnection(ConfigurationManager.ConnectionStrings("MyDb").ConnectionString);
connection.Open();
return new AdoNetUnitOfWork(connection, true);
}
}
public class AdoNetUnitOfWork : IUnitOfWork
{
public AdoNetUnitOfWork(IDbConnection connection, bool ownsConnection)
{
_connection = connection;
_ownsConnection=ownsConnection;
_transaction = connection.BeginTransaction();
}
public IDbCommand CreateCommand()
{
var command = _connection.CreateCommand();
command.Transaction = _transaction;
return command;
}
public void SaveChanges()
{
if (_transaction == null)
throw new InvalidOperationException("Transaction have already been commited. Check your transaction handling.");
_transaction.Commit();
_transaction = null;
}
public void Dispose()
{
if (_transaction != null)
{
_transaction.Rollback();
_transaction = null;
}
if (_connection != null && _ownsConnection)
{
_connection.Close();
_connection = null;
}
}
}
using (var uow = UnitOfWorkFactory.Create())
{
var repos = new UserRepository(uow);
uow.SaveChanges();
}
public class UserRepository
{
private AdoNetUnitOfWork _unitOfWork;
public UserRepository(IUnitOfWork uow)
{
if (uow == null)
throw new ArgumentNullException("uow");
_unitOfWork = uow as AdoNetUnitOfWork;
if (_unitOfWork == null)
throw new NotSupportedException("Ohh my, change that UnitOfWorkFactory, will you?");
}
public User Get(Guid id)
{
using (var cmd = _unitOfWork.CreateCommand())
{
cmd.CommandText = "SELECT * FROM Users WHERE Id = #id");
cmd.AddParameter("id", id);
// uses an extension method which I will demonstrate in a
// blog post in a couple of days
return cmd.FirstOrDefault<User>();
}
}
}
but i want to add unit of work code in my repository class. may be in AdoRepository class or StudentRepository class as a result i do not have to write the below code again and again.
using (var uow = UnitOfWorkFactory.Create())
{
var repos = new UserRepository(uow);
uow.SaveChanges();
}
so when i will be working with any repository like UserRepository then i have to repeat again and again this lines of code UnitOfWorkFactory.Create() which i do not want rather i am looking for a way to embed unit of work code in some where centralize as a result i could reduce the repeated code.
so looking for idea and suggestion. if possible give me modified version of code.
thanks
I will be straightforward about this:
A UoW that contains repositories is an anti pattern at least if DDD is involved. If it isn't, then you're dealing with a very simple app so make you life easier and use a ORM which implements what you want.
As a thumb rule, a repository may contain a UoW as an implementation detail and you should be certain that you really need a repository. A repository inside a UoW is a very specific case which is valid ONLY with CRUD apps and you'll just be reinventing a very small part of an ORM.
Using ado.net directly is wasting time (and money). Either use an ORM or a data mapper aka micro-ORM. With the latter option the UoW is the Db transaction.
You don't gain anything significant using Ado.Net,the performance gain is so small it's not even funny (I know that because I've benchmarked it). The only thing you'll get is longer devel time and human errors.
Actually, if I take a better look at your code, you're basically building a micro ORM, without knowing it. It's still not a repository though, because the abstraction doesn't have business semantics and it's too tightly coupled to an implementation detail.

Managing connection with non-buffered queries in Dapper

I have recently started using Dapper, everything seems nice and easy but there is one thing that keeps confusing me: Connection Management.
As per the documentation:
Dapper does not manage your connection's lifecycle, it assumes the
connection it gets is open AND has no existing datareaders enumerating
(unless MARS is enabled)
In light of this I started doing this inside the implementation of my repository methods:
using (var db = new SqliteConnection(connectionString)) {
// call Dapper methods here
}
Then I came across a table with a large number of records, so I though of returning an IEnumerable<T> by passing buffered: false to the Query<> method, and when I started enumerating the enumerable in the front end, boom an exception saying the connection was closed and disposed which is expected since I am wrapping my calls with the preceding using block.
Question: Best way to solve this ?
Side question: Is the way I am managing the connection the preferred way to go about it ?
I'd offer this repository pattern:
public class Repository
{
private readonly string _connectionString;
public Repository(string connectionString)
{
_connectionString = connectionString;
}
protected T GetConnection<T>(Func<IDbConnection, T> getData)
{
using (var connection = new SqlConnection(_connectionString))
{
connection.Open();
return getData(connection);
}
}
protected TResult GetConnection<TRead, TResult>(Func<IDbConnection, TRead> getData, Func<TRead, TResult> process)
{
using (var connection = new SqlConnection(_connectionString))
{
connection.Open();
var data = getData(connection);
return process(data);
}
}
}
For buffered queries you want to use first overload of GetConnection method, for non-buffered you use second, specifing callback for processing data:
public class MyRepository : Repository
{
public MyRepository(string connectionString) : base(connectionString)
{
}
public IEnumerable<MyMapObject> GetData()
{
return GetConnection(c => c.Query<MyMapObject>(query));
}
public IEnumerable<ResultObject> GetLotsOfData(Func<IEnumerable<MyMapObject>, IEnumerable<ResultObject>> process)
{
return GetConnection(c => c.Query<MyMapObject>(query, buffered: false), process);
}
}
Very basic usage:
static void Main(string[] args)
{
var repository = new MyRepository(connectionString);
var data = repository.GetLotsOfData(ProcessData);
}
public static IEnumerable<ResultObject> ProcessData(IEnumerable<MyMapObject> data)
{
foreach (var record in data)
{
var result = new ResultObject();
//do some work...
yield return result;
}
}
But keep in mind - connection may be opened for too long time in this case...
#Sergio, AWESOME! Thanks for such a great pattern. I modified it slightly to be async so that I can use it with Dapper's async methods. Makes my entire request chain async, from the controllers all the way back to the DB! Gorgeous!
public abstract class BaseRepository
{
private readonly string _ConnectionString;
protected BaseRepository(string connectionString)
{
_ConnectionString = connectionString;
}
// use for buffered queries
protected async Task<T> WithConnection<T>(Func<IDbConnection, Task<T>> getData)
{
try
{
using (var connection = new SqlConnection(_ConnectionString))
{
await connection.OpenAsync();
return await getData(connection);
}
}
catch (TimeoutException ex)
{
throw new Exception(String.Format("{0}.WithConnection() experienced a SQL timeout", GetType().FullName), ex);
}
catch (SqlException ex)
{
throw new Exception(String.Format("{0}.WithConnection() experienced a SQL exception (not a timeout)", GetType().FullName), ex);
}
}
// use for non-buffeed queries
protected async Task<TResult> WithConnection<TRead, TResult>(Func<IDbConnection, Task<TRead>> getData, Func<TRead, Task<TResult>> process)
{
try
{
using (var connection = new SqlConnection(_ConnectionString))
{
await connection.OpenAsync();
var data = await getData(connection);
return await process(data);
}
}
catch (TimeoutException ex)
{
throw new Exception(String.Format("{0}.WithConnection() experienced a SQL timeout", GetType().FullName), ex);
}
catch (SqlException ex)
{
throw new Exception(String.Format("{0}.WithConnection() experienced a SQL exception (not a timeout)", GetType().FullName), ex);
}
}
}
Use with Dapper like this:
public class PersonRepository : BaseRepository
{
public PersonRepository(string connectionString): base (connectionString) { }
// Assumes you have a Person table in your DB that
// aligns with a Person POCO model.
//
// Assumes you have an existing SQL sproc in your DB
// with #Id UNIQUEIDENTIFIER as a parameter. The sproc
// returns rows from the Person table.
public async Task<Person> GetPersonById(Guid Id)
{
return await WithConnection(async c =>
{
var p = new DynamicParameters();
p.Add("Id", Id, DbType.Guid);
var people = await c.QueryAsync<Person>(sql: "sp_Person_GetById", param: p, commandType: CommandType.StoredProcedure);
return people.FirstOrDefault();
});
}
}

How do I handle Database Connections with Dapper in .NET?

I've been playing with Dapper, but I'm not sure of the best way to handle the database connection.
Most examples show the connection object being created in the example class, or even in each method. But it feels wrong to me to reference a connection string in every clss, even if it's pulling from the web.config.
My experience has been with using a DbDataContext or DbContext with Linq to SQL or Entity Framework, so this is new to me.
How do I structure my web apps when using Dapper as my Data Access strategy?
Update: clarification from MarredCheese's comment:
"No need to use a using statement. Dapper will automatically open,
close, and dispose of the connection for you." That's not correct.
Dapper will automatically open closed connections, and it will
automatically close connections that it auto-opened, but it will not
automatically dispose of connections. Marc Gravell and Eric Lippert
both advocate using using with Dapper here.
Microsoft.AspNetCore.All: v2.0.3 | Dapper: v1.50.2
I am not sure if I am using the best practices correctly or not, but I am doing it this way, in order to handle multiple connection strings.
It's easy if you have only 1 connection string
Startup.cs
using System.Data;
using System.Data.SqlClient;
namespace DL.SO.Project.Web.UI
{
public class Startup
{
public IConfiguration Configuration { get; private set; }
// ......
public void ConfigureServices(IServiceCollection services)
{
// Read the connection string from appsettings.
string dbConnectionString = this.Configuration.GetConnectionString("dbConnection1");
// Inject IDbConnection, with implementation from SqlConnection class.
services.AddTransient<IDbConnection>((sp) => new SqlConnection(dbConnectionString));
// Register your regular repositories
services.AddScoped<IDiameterRepository, DiameterRepository>();
// ......
}
}
}
DiameterRepository.cs
using Dapper;
using System.Data;
namespace DL.SO.Project.Persistence.Dapper.Repositories
{
public class DiameterRepository : IDiameterRepository
{
private readonly IDbConnection _dbConnection;
public DiameterRepository(IDbConnection dbConnection)
{
_dbConnection = dbConnection;
}
public IEnumerable<Diameter> GetAll()
{
const string sql = #"SELECT * FROM TABLE";
// No need to use using statement. Dapper will automatically
// open, close and dispose the connection for you.
return _dbConnection.Query<Diameter>(sql);
}
// ......
}
}
Problems if you have more than 1 connection string
Since Dapper utilizes IDbConnection, you need to think of a way to differentiate different database connections.
I tried to create multiple interfaces, 'inherited' from IDbConnection, corresponding to different database connections, and inject SqlConnection with different database connection strings on Startup.
That failed because SqlConnection inherits from DbConnection, and DbConnection inplements not only IDbConnection but also Component class. So your custom interfaces won't be able to use just the SqlConnection implenentation.
I also tried to create my own DbConnection class that takes different connection string. That's too complicated because you have to implement all the methods from DbConnection class. You lost the help from SqlConnection.
What I end up doing
During Startup, I loaded all connection string values into a dictionary. I also created an enum for all the database connection names to avoid magic strings.
I injected the dictionary as Singleton.
Instead of injecting IDbConnection, I created IDbConnectionFactory and injected that as Transient for all repositories. Now all repositories take IDbConnectionFactory instead of IDbConnection.
When to pick the right connection? In the constructor of all repositories! To make things clean, I created repository base classes and have the repositories inherit from the base classes. The right connection string selection can happen in the base classes.
DatabaseConnectionName.cs
namespace DL.SO.Project.Domain.Repositories
{
public enum DatabaseConnectionName
{
Connection1,
Connection2
}
}
IDbConnectionFactory.cs
using System.Data;
namespace DL.SO.Project.Domain.Repositories
{
public interface IDbConnectionFactory
{
IDbConnection CreateDbConnection(DatabaseConnectionName connectionName);
}
}
DapperDbConenctionFactory - my own factory implementation
namespace DL.SO.Project.Persistence.Dapper
{
public class DapperDbConnectionFactory : IDbConnectionFactory
{
private readonly IDictionary<DatabaseConnectionName, string> _connectionDict;
public DapperDbConnectionFactory(IDictionary<DatabaseConnectionName, string> connectionDict)
{
_connectionDict = connectionDict;
}
public IDbConnection CreateDbConnection(DatabaseConnectionName connectionName)
{
string connectionString = null;
if (_connectDict.TryGetValue(connectionName, out connectionString))
{
return new SqlConnection(connectionString);
}
throw new ArgumentNullException();
}
}
}
Startup.cs
namespace DL.SO.Project.Web.UI
{
public class Startup
{
// ......
public void ConfigureServices(IServiceCollection services)
{
var connectionDict = new Dictionary<DatabaseConnectionName, string>
{
{ DatabaseConnectionName.Connection1, this.Configuration.GetConnectionString("dbConnection1") },
{ DatabaseConnectionName.Connection2, this.Configuration.GetConnectionString("dbConnection2") }
};
// Inject this dict
services.AddSingleton<IDictionary<DatabaseConnectionName, string>>(connectionDict);
// Inject the factory
services.AddTransient<IDbConnectionFactory, DapperDbConnectionFactory>();
// Register your regular repositories
services.AddScoped<IDiameterRepository, DiameterRepository>();
// ......
}
}
}
DiameterRepository.cs
using Dapper;
using System.Data;
namespace DL.SO.Project.Persistence.Dapper.Repositories
{
// Move the responsibility of picking the right connection string
// into an abstract base class so that I don't have to duplicate
// the right connection selection code in each repository.
public class DiameterRepository : DbConnection1RepositoryBase, IDiameterRepository
{
public DiameterRepository(IDbConnectionFactory dbConnectionFactory)
: base(dbConnectionFactory) { }
public IEnumerable<Diameter> GetAll()
{
const string sql = #"SELECT * FROM TABLE";
// No need to use using statement. Dapper will automatically
// open, close and dispose the connection for you.
return base.DbConnection.Query<Diameter>(sql);
}
// ......
}
}
DbConnection1RepositoryBase.cs
using System.Data;
using DL.SO.Project.Domain.Repositories;
namespace DL.SO.Project.Persistence.Dapper
{
public abstract class DbConnection1RepositoryBase
{
public IDbConnection DbConnection { get; private set; }
public DbConnection1RepositoryBase(IDbConnectionFactory dbConnectionFactory)
{
// Now it's the time to pick the right connection string!
// Enum is used. No magic string!
this.DbConnection = dbConnectionFactory.CreateDbConnection(DatabaseConnectionName.Connection1);
}
}
}
Then for other repositories that need to talk to the other connections, you can create a different repository base class for them.
using System.Data;
using DL.SO.Project.Domain.Repositories;
namespace DL.SO.Project.Persistence.Dapper
{
public abstract class DbConnection2RepositoryBase
{
public IDbConnection DbConnection { get; private set; }
public DbConnection2RepositoryBase(IDbConnectionFactory dbConnectionFactory)
{
this.DbConnection = dbConnectionFactory.CreateDbConnection(DatabaseConnectionName.Connection2);
}
}
}
using Dapper;
using System.Data;
namespace DL.SO.Project.Persistence.Dapper.Repositories
{
public class ParameterRepository : DbConnection2RepositoryBase, IParameterRepository
{
public ParameterRepository (IDbConnectionFactory dbConnectionFactory)
: base(dbConnectionFactory) { }
public IEnumerable<Parameter> GetAll()
{
const string sql = #"SELECT * FROM TABLE";
return base.DbConnection.Query<Parameter>(sql);
}
// ......
}
}
Hope all these help.
It was asked about 4 years ago... but anyway, maybe the answer will be useful to someone here:
I do it like this in all the projects.
First, I create a base class which contains a few helper methods like this:
public class BaseRepository
{
protected T QueryFirstOrDefault<T>(string sql, object parameters = null)
{
using (var connection = CreateConnection())
{
return connection.QueryFirstOrDefault<T>(sql, parameters);
}
}
protected List<T> Query<T>(string sql, object parameters = null)
{
using (var connection = CreateConnection())
{
return connection.Query<T>(sql, parameters).ToList();
}
}
protected int Execute(string sql, object parameters = null)
{
using (var connection = CreateConnection())
{
return connection.Execute(sql, parameters);
}
}
// Other Helpers...
private IDbConnection CreateConnection()
{
var connection = new SqlConnection(...);
// Properly initialize your connection here.
return connection;
}
}
And having such a base class I can easily create real repositories without any boilerplate code:
public class AccountsRepository : BaseRepository
{
public Account GetById(int id)
{
return QueryFirstOrDefault<Account>("SELECT * FROM Accounts WHERE Id = #Id", new { id });
}
public List<Account> GetAll()
{
return Query<Account>("SELECT * FROM Accounts ORDER BY Name");
}
// Other methods...
}
So all the code related to Dapper, SqlConnection-s and other database access stuff is located in one place (BaseRepository). All real repositories are clean and simple 1-line methods.
I hope it will help someone.
I created extension methods with a property that retrieves the connection string from configuration. This lets the callers not have to know anything about the connection, whether it's open or closed, etc. This method does limit you a bit since you're hiding some of the Dapper functionality, but in our fairly simple app it's worked fine for us, and if we needed more functionality from Dapper we could always add a new extension method that exposes it.
internal static string ConnectionString = new Configuration().ConnectionString;
internal static IEnumerable<T> Query<T>(string sql, object param = null)
{
using (SqlConnection conn = new SqlConnection(ConnectionString))
{
conn.Open();
return conn.Query<T>(sql, param);
}
}
internal static int Execute(string sql, object param = null)
{
using (SqlConnection conn = new SqlConnection(ConnectionString))
{
conn.Open();
return conn.Execute(sql, param);
}
}
I do it like this:
internal class Repository : IRepository {
private readonly Func<IDbConnection> _connectionFactory;
public Repository(Func<IDbConnection> connectionFactory)
{
_connectionFactory = connectionFactory;
}
public IWidget Get(string key) {
using(var conn = _connectionFactory())
{
return conn.Query<Widget>(
"select * from widgets with(nolock) where widgetkey=#WidgetKey", new { WidgetKey=key });
}
}
}
Then, wherever I wire-up my dependencies (ex: Global.asax.cs or Startup.cs), I do something like:
var connectionFactory = new Func<IDbConnection>(() => {
var conn = new SqlConnection(
ConfigurationManager.ConnectionStrings["connectionString-name"];
conn.Open();
return conn;
});
Best practice is a real loaded term. I like a DbDataContext style container like Dapper.Rainbow promotes. It allows you to couple the CommandTimeout, transaction and other helpers.
For example:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using Dapper;
// to have a play, install Dapper.Rainbow from nuget
namespace TestDapper
{
class Program
{
// no decorations, base class, attributes, etc
class Product
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public DateTime? LastPurchase { get; set; }
}
// container with all the tables
class MyDatabase : Database<MyDatabase>
{
public Table<Product> Products { get; set; }
}
static void Main(string[] args)
{
var cnn = new SqlConnection("Data Source=.;Initial Catalog=tempdb;Integrated Security=True");
cnn.Open();
var db = MyDatabase.Init(cnn, commandTimeout: 2);
try
{
db.Execute("waitfor delay '00:00:03'");
}
catch (Exception)
{
Console.WriteLine("yeah ... it timed out");
}
db.Execute("if object_id('Products') is not null drop table Products");
db.Execute(#"create table Products (
Id int identity(1,1) primary key,
Name varchar(20),
Description varchar(max),
LastPurchase datetime)");
int? productId = db.Products.Insert(new {Name="Hello", Description="Nothing" });
var product = db.Products.Get((int)productId);
product.Description = "untracked change";
// snapshotter tracks which fields change on the object
var s = Snapshotter.Start(product);
product.LastPurchase = DateTime.UtcNow;
product.Name += " World";
// run: update Products set LastPurchase = #utcNow, Name = #name where Id = #id
// note, this does not touch untracked columns
db.Products.Update(product.Id, s.Diff());
// reload
product = db.Products.Get(product.Id);
Console.WriteLine("id: {0} name: {1} desc: {2} last {3}", product.Id, product.Name, product.Description, product.LastPurchase);
// id: 1 name: Hello World desc: Nothing last 12/01/2012 5:49:34 AM
Console.WriteLine("deleted: {0}", db.Products.Delete(product.Id));
// deleted: True
Console.ReadKey();
}
}
}
Try this:
public class ConnectionProvider
{
DbConnection conn;
string connectionString;
DbProviderFactory factory;
// Constructor that retrieves the connectionString from the config file
public ConnectionProvider()
{
this.connectionString = ConfigurationManager.ConnectionStrings[0].ConnectionString.ToString();
factory = DbProviderFactories.GetFactory(ConfigurationManager.ConnectionStrings[0].ProviderName.ToString());
}
// Constructor that accepts the connectionString and Database ProviderName i.e SQL or Oracle
public ConnectionProvider(string connectionString, string connectionProviderName)
{
this.connectionString = connectionString;
factory = DbProviderFactories.GetFactory(connectionProviderName);
}
// Only inherited classes can call this.
public DbConnection GetOpenConnection()
{
conn = factory.CreateConnection();
conn.ConnectionString = this.connectionString;
conn.Open();
return conn;
}
}
Everyone appears to be opening their connections entirely too early? I had this same question, and after digging through the Source here - https://github.com/StackExchange/dapper-dot-net/blob/master/Dapper/SqlMapper.cs
You will find that every interaction with the database checks the connection to see if it is closed, and opens it as necessary. Due to this, we simply utilize using statements like above without the conn.open(). This way the connection is opened as close to the interaction as possible. If you notice, it also immediately closes the connection. This will also be quicker than it closing automatically during disposal.
One of the many examples of this from the repo above:
private static int ExecuteCommand(IDbConnection cnn, ref CommandDefinition command, Action<IDbCommand, object> paramReader)
{
IDbCommand cmd = null;
bool wasClosed = cnn.State == ConnectionState.Closed;
try
{
cmd = command.SetupCommand(cnn, paramReader);
if (wasClosed) cnn.Open();
int result = cmd.ExecuteNonQuery();
command.OnCompleted();
return result;
}
finally
{
if (wasClosed) cnn.Close();
cmd?.Dispose();
}
}
Below is a small example of how we use a Wrapper for Dapper called the DapperWrapper. This allows us to wrap all of the Dapper and Simple Crud methods to manage connections, provide security, logging, etc.
public class DapperWrapper : IDapperWrapper
{
public IEnumerable<T> Query<T>(string query, object param = null, IDbTransaction transaction = null, bool buffered = true, int? commandTimeout = null, CommandType? commandType = null)
{
using (var conn = Db.NewConnection())
{
var results = conn.Query<T>(query, param, transaction, buffered, commandTimeout, commandType);
// Do whatever you want with the results here
// Such as Security, Logging, Etc.
return results;
}
}
}
I wrap connection with the helper class:
public class ConnectionFactory
{
private readonly string _connectionName;
public ConnectionFactory(string connectionName)
{
_connectionName = connectionName;
}
public IDbConnection NewConnection() => new SqlConnection(_connectionName);
#region Connection Scopes
public TResult Scope<TResult>(Func<IDbConnection, TResult> func)
{
using (var connection = NewConnection())
{
connection.Open();
return func(connection);
}
}
public async Task<TResult> ScopeAsync<TResult>(Func<IDbConnection, Task<TResult>> funcAsync)
{
using (var connection = NewConnection())
{
connection.Open();
return await funcAsync(connection);
}
}
public void Scope(Action<IDbConnection> func)
{
using (var connection = NewConnection())
{
connection.Open();
func(connection);
}
}
public async Task ScopeAsync<TResult>(Func<IDbConnection, Task> funcAsync)
{
using (var connection = NewConnection())
{
connection.Open();
await funcAsync(connection);
}
}
#endregion Connection Scopes
}
Examples of usage:
public class PostsService
{
protected IConnectionFactory Connection;
// Initialization here ..
public async Task TestPosts_Async()
{
// Normal way..
var posts = Connection.Scope(cnn =>
{
var state = PostState.Active;
return cnn.Query<Post>("SELECT * FROM [Posts] WHERE [State] = #state;", new { state });
});
// Async way..
posts = await Connection.ScopeAsync(cnn =>
{
var state = PostState.Active;
return cnn.QueryAsync<Post>("SELECT * FROM [Posts] WHERE [State] = #state;", new { state });
});
}
}
So I don't have to explicitly open the connection every time.
Additionally, you can use it this way for the convenience' sake of the future refactoring:
var posts = Connection.Scope(cnn =>
{
var state = PostState.Active;
return cnn.Query<Post>($"SELECT * FROM [{TableName<Post>()}] WHERE [{nameof(Post.State)}] = #{nameof(state)};", new { state });
});
What is TableName<T>() can be found in this answer.
Hi #donaldhughes I'm new on it too, and I use to do this:
1 - Create a class to get my Connection String
2 - Call the connection string class in a Using
Look:
DapperConnection.cs
public class DapperConnection
{
public IDbConnection DapperCon {
get
{
return new SqlConnection(ConfigurationManager.ConnectionStrings["Default"].ToString());
}
}
}
DapperRepository.cs
public class DapperRepository : DapperConnection
{
public IEnumerable<TBMobileDetails> ListAllMobile()
{
using (IDbConnection con = DapperCon )
{
con.Open();
string query = "select * from Table";
return con.Query<TableEntity>(query);
}
}
}
And it works fine.

Categories

Resources