Develop Reference

C# work with database which type is not known in advance

I have an app that collects data and writes it to a database. The database type is not known in advance, it's defined via an .ini file. So I have a method like this, if the database is Firebird SQL:
public bool writeToDB()
{
FbConnection dbConn = new FbConnection(connString);
dbConn.Open();
FbTransaction dbTrans = dbConn.BeginTransaction();
FbCommand writeCmd = new FbCommand(cmdText, dbConn, dbTrans);
/* some stuff */
writeCmd.ExecuteNonQuery();
dbTrans.Commit();
writeCmd.Dispose();
dbConn.Close();
return true;
}
To make the same work for e.g. MS Access database, I only have to replace FbConnection, FbTransaction and FbCommand with OleDbConnection, OleDbTransaction and OleDbCommand respectively.
But I don't want to have a separate identical method for each type of database.
Is it possible to define the database connection / transaction / command type at runtime, after the database type is known?
Thanks

When you're writing code at this level - opening and closing connections, creating and executing commands - there's probably no benefit in trying to make this method or class database-agnostic. This code is about implementation details so it makes sense that it would be specific to an implementation like a particular database.
But I don't want to have a separate identical method for each type of database.
You're almost certainly better off having separate code for separate implementations. If you try to write code that accommodates multiple implementations it will be complicated. Then another implementation (database) comes along which almost but doesn't quite fit the pattern you've created and you have to make it even more complicated to fit that one in.
I don't know the specifics of what you're building, but "what if I need a different database" is usually a "what if" that never happens. When we try to write one piece of code that satisfies requirements we don't actually have, it becomes complex and brittle. Then real requirements come along and they're harder to meet because our code is tied in knots to do things it doesn't really need to do.
That doesn't mean that all of our code should be coupled to a specific implementation, like a database. We just have to find a level of abstraction that's good enough. Does our application need to interact with a database to save and retrieve data? A common abstraction for that is a repository. In C# we could define an interface like this:
public interface IFooRepository
{
Task<Foo> GetFoo(Guid fooId);
Task Save(Foo foo);
}
Then we can create separate implementations for different databases if and when we need them. Code that depends on IFooRepository won't be coupled to any of those implementations, and those implementations won't be coupled to each other.

First (and Second and Third). STOP REINVENTING THE WHEEL.
https://learn.microsoft.com/en-us/ef/core/providers/?tabs=dotnet-core-cli
Alot of code and an alot of better testing has already been done.
Guess what is in that larger list:
FirebirdSql.EntityFrameworkCore.Firebird Firebird 3.0 onwards
EntityFrameworkCore.Jet Microsoft Access files
......
So I'm gonna suggest something in lines with everyone else. BUT also... allows for some reuse.
I am basing this .. on the fact the Entity Framework Core...provides functionality to several RDBMS.
See:
https://learn.microsoft.com/en-us/ef/core/providers/?tabs=dotnet-core-cli
public interface IEmployeeDomainDataLayer
{
Task<Employee> GetSingle(Guid empKey);
Task Save(Employee emp);
}
public abstract class EmployeeEntityFrameworkDomainDataLayerBase : IEmployeeDomainDataLayer
{
/* you'll inject a MyDbContext into this class */
//implement Task<Employee> GetSingle(Guid empKey); /* but also allow this to be overrideable */
//implement Task Save(Employee emp); /* but also allow this to be overrideable */
}
public class EmployeeJetEntityFrameworkDomainDataLayer : EmployeeEntityFrameworkDomainDataLayerBase, IEmployeeDomainDataLayer
{
/* do not do any overriding OR override if you get into a jam */
}
public class EmployeeSqlServerEntityFrameworkDomainDataLayer : EmployeeEntityFrameworkDomainDataLayerBase, IEmployeeDomainDataLayer
{
/* do not do any overriding OR override if you get into a jam */
}
You "code to an interface, not an implementation". Aka, your business layer codes to IEmployeeDomainDataLayer.
This gives you most code in EmployeeEntityFrameworkDomainDataLayerBase. BUT if any of the concretes give you trouble, you have a way to code something up ONLY FOR THAT CONCRETE.
If you want DESIGN TIME "picking of the RDBMS", then you do this:
You inject one of the concretes ( EmployeeJetEntityFrameworkDomainDataLayer OR EmployeeSqlServerEntityFrameworkDomainDataLayer ) into your IOC, based on which backend you want to wire to.
If you want RUN-TIME "picking of the RDMBS", you can define a "factory".
public static class HardCodedEmployeeDomainDataLayerFactory
{
public static IEmployeeDomainDataLayer getAnIEmployeeDomainDataLayer(string key)
{
return new EmployeeJetEntityFrameworkDomainDataLayer();
// OR (based on key)
return new EmployeeSqlServerEntityFrameworkDomainDataLayer();
}
}
The factory above suffers from IOC anemia. Aka, if your concretes need items for their constructors..you have to fudge them.
A better idea of the above is the kissing cousin of "Factory" pattern, called the Strategy Design.
It is a "kinda factory", BUT you inject the possible results of the "factory" in via a constructor. Aka, the "factory" is NOT hard coded...and does NOT suffer from IOC anemia.
See my answer here:
Using a Strategy and Factory Pattern with Dependency Injection

Refactor over C# code to implement some kind of Factory Design Pattern

I have a Web API App (.NET Core) that is part of my backend. I have implemented the Unit of Work and Repository patterns into a N-Layer 1 Tier Architecture. The way how the Client App and the backend communicate is through sending DTO's (Data Transfer Objects). So, I have different DTO's obviously with a different structure and composition according to what the Repository Method is requesting. This is because at design time I define the type of DTO the Client is expecting, something like this:
public EmployeeDto GetEMployeeDto()
{
public EmployeeDto employee = new EmployeeDto();
.
.
.
// Code to get data from data base and full fill the fields of the DTO.
.
.
.
return EmployeeDto;
}
Where EmployeeDto is something like this:
public cladd EmployeeDto()
{
public int Employee Id {get;set;}
public string EmployeeName {get;set;}
}
What I would like to do is remove the instantiating code (public EmployeeDto employee = new EmployeDto();) which is tight coupling, and implement some kind of factory pattern where I just call the Factory object, pass the type of DTO I want (because I already know it) and that the Factory returns me the type of DTO I requested and with the DTO state (data).
Just keep on mind that I have several types of DTO's with different structure, so the way how they are fill is different and that the Factory object could be called in different methods and could be requested different DTO's.
Something like:
var employee = FactoryDto.GetDto(EmployeDto);
var employeeHistory = FactoryDto.GetDto(EmployeeHistoryDto);
With just one line get the whole "package" the DTO itself and the data.
I just need a guidance to which Design Pattern could be the best option and an example, I have read about the Creational Design pattern Factory and the different flavors it has but I have not understood yet how could this achieve.
This is one of the approaches I have:
public EmployeeDto GetEmployeeDto(string dtoType)
{
var factoryDto = new FactoryDto();
var empDto = factoryDto.GetDto(dtoType);
return (EmployeeDto)empDto;
}
public class FactoryDto
{
public object GetDto(string dtoType)
{
switch(dtoType)
{
case "EmployeeDto":
{
return new EmployeeDto();
}
case "EmployeeHistoryDto":
{
return new EmployeeHistoryDto();
}
default:
{
return new EmployeeHistoryDto();
}
}
}
}
But, I do not think this is a correct answer, casting the Dto when returning the Dto and I do not like that Object generic type in the Factory class. Is there a better way to improve this approach using any of the Creational Design Patterns? Regards!

There is no way (at least not one that makes sense to me) to decouple an implementation meant to deal with a specific type, from this type. It seems perfectly natural for a repository dealing with EmployeeDto type to know directly about this type.
Not sure how does your design look at a broader view, but perhaps what you are missing is a generic interface, so your client side is not much coupled? For example, something like this:
public interface IRepository<T>
{
IEnumerable<T> GetAll<T>();
T Get<T>(int id);
}
An implication is that you would not have GetEmployeeDto method as in your example, but just a general GetDto or even Get. Which leads me to another hint: following DDD-related patterns, a repository should return a domain object that is ready to use, not a DTO.
This way your domain layer can just use the repository and focus on domain logic, instead of being forced to deal with conversions. As mentioned in the comments it's good to configure an automapping library to do this for you anyway. But even with an automapper, all technicalities should be hidden from the domain layer.

Database layer design with unit tests in mind

Basically I need a bunch of database access interfaces that can be implemented differently, or to put it another way, de-coupled from the underlying actual database:
interface IDbConnexion
{
IDbCollection Collection { get; }
Task Connect();
}
interface IDbCollection
{
void Insert(Model model);
}
I will use Collection to do CRUDs. But the problem with that is, for a concrete implementation for a particular database, eg MongoDB, I would need to do the following:
class MongoDbCollection: IDbCollection
{
IMongoCollection<Model> _collection;
void IDbCollection.Insert(Model model)
{
_collection.InsertOne(model);
}
}
Because this is my own wrapper for the mongodb client, I would need to wrap every single method provided by IMongoCollection so that I can use its full power. Eg. if I wanna use InsertMany then I would need to add my own wrapper for InsertMany to MongoDbCollection. And as my need grows, MongoDbCollection will inflate too. But I don't wanna do this. I should just be able to access all the methods the mongodb driver exposes to me without too much extra work.
So I am thinking an alternative:
interface IDbConnexion
{
IMongoCollection<Model> Collection { get; }
...
}
But the problem with this is that my interface is coupled with mongodb. My unit test mockup will need to implement IMongoCollection just for the sake of making it compile.
So my question is: is there a design pattern to allow me full access to the concrete Collection property while allowing me to easily change the concrete implementation.

Where should I put commonly used data access code with logic not fitting to Repository when using Service classes on top of Repository/UnitOrWork?

In my earlier question I was asking about implementing repository/unit of work pattern for large applications built with an ORM framework like EF.
One followup problem I cannot come through right now is where to put codes containing business logic, but still lower-level enough to be used commonly in many other part of the application.
For example here is a few such method:
Getting all users in one or more roles.
Getting all cities where a user has privileges within an optional
region.
Getting all measure devices of a given device type, within a given
region for which the current user has privileges.
Finding a product by code, checking if it's visible and throwing
exception if not found or not visible.
All of these methods use a UnitOfWork for data access or manipulation, and receive several parameters as in their specification. I think everyone could write a lot more example for such common tasks in a large project. My question is where shall I put tese method implementations? I can see the following options currently.
Option 1: Every method goes to its own service class
public class RegionServices {
// support DI constructor injection
public RegionServices(IUnitOfWork work) {...}
...
public IEnumerable<City> GetCitiesForUser(User user, Region region = null) { ... }
...
}
public class DeviceServices {
// support DI constructor injection
public DeviceServices(IUnitOfWork work) {...}
...
public IEnumerable<Device> GetDevicesForUser(User user, DeviceType type, Region region = null) { ... }
...
}
What I don't like about it is that if a higher-level application service needs to call for example 3 or these methods, then it needs to instantiate 3 services, and if I use DI then I even have to put all 3 into the constructor, easily resulting quite a bit of code smell.
Option 2: Creating some kind of Facade for such common data access
public class DataAccessHelper {
// support DI constructor injection
public DataAccessHelper(IUnitOfWork work) {...}
...
public IEnumerable<City> GetCitiesForUser(User user, Region region = null) { ... }
public IEnumerable<Device> GetDevicesForUser(User user, DeviceType type, Region region = null) { ... }
public IEnumerable<User> GetUsersInRoles(params string[] roleIds) { ... }
...
}
I don't like it because it feels like violating the SRP, but its usage can be much more comfortable however.
Option 3: Creating extension methods for the Repositories
public static class DataAccessExtensions {
public static IEnumerable<City> GetCitiesForUser(this IRepository repo, User user, Region region = null) { ... }
}
Here IRepository is an interface with generic methods like Query<T>, Save<T>, etc. I don't like it either because it feels like I want to give business logic to repositories which is not advisable AFAIK. However, it expresses that these methods are common and lower level than service classes, which I like.
Maybe there are other options as well?... Thank you for the help.

If you say that a certain piece of domain logic needs to look at 3 distinct pieces of information in order to make a decision then we will need to provide this information to it.
Further if we say that each of these distinct pieces can be useful to other parts of the domain then each of them will need to be in its own method also. We can debate whether each query needs to be housed in a separate class or not depending on your domain/design.
The point I wanted to make is that there will be a application service which delegates to one or more Finder classes (classes where your queries are housed), these classes house only queries and then accumulate the results and pass it down to a Domain Service as method params.
The domain service acts on on the received parameters executes the logic and returns the result. This way the domain service is easily testable.
psuedo code
App Service
result1 = finder.query1()
result2 = finder.query2()
result3= yetanotherfinder.query();
domainresult = domainservice.calculate(result1,result2,result3);

Repositories belong to the domain, queries do not (http://www.jefclaes.be/2014/01/repositories-where-did-we-go-wrong_26.html).
You could define explicit queries and query handlers and use those outside of your domain.
public class GetUserStatisticsQuery
{
public int UserId { get; set; }
}
public class GetUserStatisticsQueryResult
{
...
}
public class GetUserStatisticsQueryHandler :
IHandleQuery<GetUserStatisticsQuery, GetUserStatisticsQueryResult>
{
public GetUserStatisticsQueryResult Handle(GetUserStatisticsQuery query)
{
... "SELECT * FROM x" ...
}
}
var result = _queryExecutor.Execute<GetUserStatisticsQueryResult>(
new GetUserStatisticsQuery(1));

I'm adding my conclusion as an answer, because I quickly realized that this question is quite relative and not exact, heavily depends on personal favours or design trends.
The comments and the answers helped me in seeing more clearly how things like this should basically be implemented, thank you for all of your effort.
Conclusion
A "repository" should be responsible clearly only for data persisting. Because it doesn't hold any domain logic, or type specific logc, I think it can be represented and implemented as an IRepository interface with generic methods like Save<T>, Delete<T>, Query<T>, GetByID<T>, etc. Please refer to my previous question mentioned in the beginning of my original post.
On the other hand, I think (at least now with my current project) that introducing new class/classes for each lower-level domain logic (in the most cases some kind of querying logic) task is a bit over-engineered solution, which is not needed for me. I mean I don't want to introduce classes like GetUsersInRoles or GetDevicesInRegionWithType, etc. I feel I would end up with a lot of classes, and a lot of boilerplate code when refering them.
I decided to implement the 3rd option, adding static query functions as extensions to IRepository. It can be nicely separated in a Queries project folder, and structured in several static classes each named after the underlying domain model on which it defines operations. For example I've implemented user related queries as follows: in Queries folder I've created a UserQueries.cs file, in which I have:
public static class UserQueries {
public static IEnumerable<User> GetInRoles(this IRepository repository, params string[] roles)
{
...
}
}
This way I can easily and comfortable access such methods via extensions on every IRepository, the methods are unit-testable and support DI (as they are callable on any IRepository implementation). This technique fits best for my current needs.
It can be refactored even further to make it even cleaner. I could introduce "ghost" sealed classes like UserQueriesWrapper and use it to structure the calling code and this way not put every kind of such extensions to IRepository. I mean something like this:
// technical class, wraps an IRepository dummily forwarding all members to the wrapped object
public class RepositoryWrapper : IRepository
{
internal RepositoryWrapper(IRepository repository) {...}
}
// technical class for holding user related query extensions
public sealed class UserQueriesWrapper : RepositoryWrapper {
internal UserQueriesWrapper(IRepository repository) : base(repository) {...}
}
public static class UserQueries {
public static UserQueriesWrapper Users(this IRepository repository) {
return new UserQueriesWrapper(repository);
}
public static IEnumerable<User> GetInRoles(this UserQueriesWrapper repository, params string[] roles)
{
...
}
}
...
// now I can use it with a nicer and cleaner syntax
var users = _repo.Users().GetInRoles("a", "b");
...
Thank you for the answers and comments again, and please if there is something I didn't notice or any gotcha with this technique, leave a comment here.

Constructive criticism on this class

I've just reviewed some code that looked like this before
public class ProductChecker
{
// some std stuff
public ProductChecker(int AccountNumber)
{
var account = new AccountPersonalDetails(AccountNumber);
//Get some info from account and populate class fields
}
public bool ProductACriteriaPassed()
{
//return some criteria based on stuff in account class
//but now accessible in private fields
}
}
There has now been some extra criteria added which needs data not in the AccountPersonalDetails class
the new code looks like this
public class ProductChecker
{
// some std stuff
public ProductChecker(int AccountNumber)
{
var account = new AccountPersonalDetails(AccountNumber);
var otherinfo = getOtherInfo(AccountNumber)
//Get some info from account and populate class fields
}
public bool ProductACriteriaPassed()
{
//return some criteria based on stuff in account class
// but now accessible in private fields and other info
}
public otherinfo getOtherInfo(int AccountNumber)
{
//DIRECT CALL TO DB TO GET OTHERINFO
}
}
I'm bothered by the db part but can people spell out to me why this is wrong? Or is it?

In a layered view of your system, it looks like ProductChecker belongs to the business rules / business logic layer(s), so it shouldn't be "contaminated" with either user interaction functionality (that belongs in the layer(s) above) or -- and that's germane to your case -- storage functionality (that belongs in the layer(s) below).
The "other info" should be encapsulated in its own class for the storage layers, and that class should be the one handling persist/retrieve functionality (just like I imagine AccountPersonalDetails is doing for its own stuff). Whether the "personal details" and "other info" are best kept as separate classes or joined into one I can't tell from the info presented, but the option should be critically considered and carefully weighed.
The rule of thumb of keeping layers separate may feel rigid at times, and it's often tempting to shortcut it to add a feature by miscegenation of the layers -- but to keep your system maintainable and clean as it grows, I do almost invariably argue for layer separation whenever such a design issue arises. In OOP terms, it speaks to "strong cohesion but weak coupling"; but in a sense it's more fundamental than OOP since it also applies to other programming paradigms, and mixes thereof!-)

It seems like the extra data grabbed in getOtherInfo should be encapsulated as part of the AccountPersonalDetails class, and thus already part of your account variable in the constructor when you create a new AccountPersonalDetails object. You pass in AccountNumber to both, so why not make AccountPersonalDetails gather all the info you need? Then you won't have to tack on extra stuff externally, as you're doing now.

It definitely looks like there might be something going haywire with the design of the class...but it's hard to tell without knowing the complete architecture of the application.
First of all, if the OtherInfo object pertains to the Account rather than the Product you're checking on...it's introducing responsibilities to your class that shouldn't be there.
Second of all, if you have a Data Access layer...then the ProductChecker class should be using the Data Access layer to retrieve data from the database rather than making direct calls in to retrieve the data it needs.
Third of all, I'm not sure that the GetOtherInfo method needs to be public. It looks like something that should only be used internally to your class (if, in fact, it actually belongs there to begin with). In that case, you also shouldn't need to pass around the accountId (you class should hold that somewhere already).
But...if OtherInfo pertains to the Product you're checking on AND you have no real Data Access layer then I can see how this might be a valid design.
Still, I'm on your side. I don't like it.

considering that an accountNumber was passed into the constructor you shouldn't have to pass it to another method like that.

A few points
The parameter names are pascal case, instead of camel (this maybe a mistake)
getOtherInfo() looks like it's a responsibility of AccountPersonalDetails and so should be in that class
You may want to use a Façade class or Repository pattern to retrieve your AccountPersonalDetails instead of using a constructor
getOtherInfo() may also be relevant for this refactor, so the database logic isn't embedded inside the domain object, but in a service class (the Façade/Repository)
ProductACriteriaPassed() is in the right place

I would recommend this:
public class AccountPersonalDetails
{
public OtherInfo OtherInfo { get; private set; }
}
public class ProductChecker
{
public ProductChecker(AccountPersonalDetails) {}
}
// and here's the important piece
public class EitherServiceOrRepository
{
public static AccountPersonalDetails GetAccountDetailsByNumber(int accountNumber)
{
// access db here
}
// you may also feel like a bit more convinience via helpers
// this may be inside ProductCheckerService, though
public static ProductChecker GetProductChecker(int accountNumber)
{
return new ProductChecker(GetAccountDetailsByNumber(accountNumber));
}
}
I'm not expert in Domain-Driven Design but I believe this is what DDD is about. You keep your logic clean of DB concerns, moving this to external services/repositories. Will be glad if somebody correct me if I'm wrong.

Whats good. It looks like you have a productChecker with a nice clear purpose. Check products. You'd refactor or alter this because your have a need to. If you don't need to, you wouldn't. Here's what I would probably do.
It "feels" clunky to create a new instance of the class for each account number. A constructor argument should be something required for the class to behave correctly. Its a parameter of the class, not a dependency. It leads to the tempation to do a lot of work in the constructor. Usage of the class should look like this:
result = new ProductChecker().ProductACriteriaPassed(accountNumber)
Which I'd quickly rename to indicate it does work.
result = new ProductChecker().PassesProductACriteria(accountNumber)
A few others have mentioned that you may want to split out the database logic. You'd want to do this if you want unit tests that are fast. Most programs want unit tests (unless you are just playing around), and they are nicer if they are fast. They are fast when you can get the database out of the way.
Let's make a dummy object representing results of the database, and pass it to a method that determines whether the product passes. If not for testibility, this would be a private. Testability wins. Suppose I want to verify a rule such as "the product must be green if the account number is prime." This approach to unit testing works great without fancy infrastructure.
// Maybe this is just a number of items.
DataRequiredToEvaluateProduct data = // Fill in data
// Yes, the next method call could be static.
result = new ProductChecker().CheckCriteria(accountNumber, data)
// Assert result
Now we need to connect the database. The database is a dependency, its required for the class to behave correctly. It should be provided in the constructor.
public class ProductRepository {} // Define data access here.
// Use the ProductChecker as follows.
result = new ProductChecker(new ProductRepository()).CheckCriteria(accountNumber)
If the constructor gets annoyingly lengthy (it probably has to read a config file to find the database), create a factory to sort it out for you.
result = ProductCheckerFactory().GimmeProductChecker().CheckCriteria(accountNumber)
So far, I haven't used any infrastructure code. Typically, we'd make the above easier and prettier with mocks and dependency injection (I use rhinomocks and autofac). I won't go into that. That is only easier if you already have it in place.