I want to code an application for both SQL Server and SQLite, but not have to duplicate all database access (that would clearly be unworkable!).
Is there any way of creating a class where the DB access is handled as required, but presented to the rest of the application as set of more generic common objects (i.e. a DataSet, a DataTable, etc.) irrespective of which DB the data was retrieved from?
Yes, you have some pretty good starting points in the .NET framework. Almost all database actions are abstracted away in interfaces, like IDbConnection, IDbCommand, IDataReader, etc. It is pretty easy to do if you rely on the interfaces (I have done it for our company supporting a lot of database platforms).
There are two common problems you have to tackle:
Specifying and creating of the connection (in our case find and load some drivers too). The DbProviderFactory can be of help;
Different database implementations of SQL. Consensus: rely on the shared subset only, or create an intermediary layer that abstracts this away.
We have written our own, but there are also frameworks that do this all for you already, like Entity Framework.
Related
I have a project I developed with C# windows forms using SQL Server as database but I want to make the project standalone without any database server. I found SQLite as an option to go for, Is it possible to migrate my SQL Server database to SQLite and not affecting my code? And how do I go about it?
I used entityframework code first in connecting the SQL Server database
The answer is almost certainly going to be "yes." Depending on a few things, you might have to change very little (or no) code, or you might have to change a lot.
The first consideration is your SQL code. If you were very careful to write ANSI-compliant SQL and you didn't use any of the built-in SQL Server views or T-SQL-specific functions, you may not have to re-write much code at all. In reality, you probably will have to at least write some. In particular, while SQL Server's engine is meant to handle multiple concurrent sessions and queries, SQLite is not: you will need to manage your program carefully to ensure no two threads attempt to access the SQLite database at once.
The second consideration is how your application calls the database. Again, depending on your design, you may need to re-write almost no code, or you may need to re-write a lot. In my C# applications, I create an interface for database providers that defines common functionality (select, delete, insert, etc). Then I create simple wrapper classes for different RDBMS that implement the interface. When I need to switch databases, I simply instantiate and use a different class. If you have your project setup like this, then you'd simply need to create a new class for SQLite that implements your database interface and instantiate that instead of your SQL Server class. If you wrote a lot of SQL Server specific C# code into your business logic, you might have a lot of coding to do.
I have read many posts concerning the issue of having several databases and how to design a DAL efficiently in this case. In many cases, the forum suggests to apply the repository pattern, which works fine in most cases.
However, I find myself in a different situation. I have 3 different databases: Oracle, OLE DB and SQLServer. Currently, there exists a unique DAL with many different classes sending SQL queries down to a layer below to be executed in the corresponding database. They way the system works is that two of the databases are only used to read information from them, and the other one is used to either store this same information or read it later on. I have to propose a better design to the current implementation, but it seems as if a common interface for all three databases is not plausible from an architectural point of view.
Is there any design pattern that solves this situation? Should I have three different DALs? Or perhaps it is possible (and advisable) to apply the repository pattern to this problem?
Answers to your question will probably be very subjective. These are some thoughts.
You could apply command-query separation. The query side integrates directly with your data layer, bypassing any business or domain layer, and the returning entities are optimized for read and project from your databases. This layer could also be responsible to merge results from different database calls.
The command side consists of command handlers, using domain or business entities, which are mapped from your R/W database.
By doing this, the interface that you expose will be more clear and business oriented.
I'm not sure that completely abstracting out the data access layer with custom units of work and repositories is really needed: do the advantages outweigh the disadvantages? They rarely do, because you will you ever change a database technology? And if you do, this probably means a rewrite anyway. Also, if you use entity framework code first, you already have unit of work and an abstraction on top of your database; and the flexibility of using LINQ.
Bottom line - try not to over-engineer/over-abstract things; or make things super-generic.
Your core/business code should never be dependent on any contract/interface/class that is placed in the DAL layer of the application.
Accessing data is something the business/core layer of your application needs to be able to do, and this it should be able to do without any dependency of SQL statements and without having any knowledge of the underlying data access technology.
I think you need to remove any "sql" statements from the core part of the application. SQL is vendor dependent and any dependency to a specific database engine needs to be clean out of you core, and moved to the DAL where it belongs. Then you need to create interfaces that resides outside of the DAL(s) which you then create implementation classes for in one or many DAL modules/classes. Your DAL can be dependent of your core, but not the other way around.
I don't see why the repository layer can't be used in this case. When I have a database which I can only read from, I usually let the name of the repository interface indicate this, like ICompanyRepositoryRead.
I am working on a .NET web api service(with Odata support) to support Mobile client.The service should support both Oracle and SQL server databases, but only one database type will be used at a time, according to which ever database technology client is using.
How to create database agnostic data access layer? Dont want to write code twice - once for SQL server and once for Oracle.
Also it seems like in order to support oracle in EF, 3rd party oracle drivers are required - either from devart or oracle's ODP.NET.
I am debating should I use old style ADO.NET or use EF for building data access layer.
I will appreciate any help on this.
Thanks!
Your question seems to revolve around multiple concerns, i'll give answers based on my views on them:
1.- ¿How can you create a Database (DB Engine) agnostic DAL?
A: One approach for this is to follow the Repository pattern and/or use interfaces to decouple the code that manipulates the data from the code that retrieves/inserts it. The actual implementation of the interfaces used by your code to get the data can also be taylored to be DB Engine agnostic, if you're going to use ADO.NET, you can check out the Enterprise Library for some very useful code which is DB Engine agnostic. Entity Framework is also compatible with different DB engines but, as you mentioned, can only interact with one DB at a time, so whenever you generate the model, you tie it to the specifics of the DB Engine that your DB is hosted in. This is related to another concern in your question:
2.- ¿Should you use plain old ADO.NET or EF?
A: This is a very good question, which i'm sure has been asked before many times and given that both approaches give you the same practical results of being able to retrieve and manipulate data, the resulting question is: ¿what is your personal preference for coding and the time/resources constraints of the project?
IMO, Entity Framework is best suited for Code-First projects and when your business logic doesn't require complex logging, transactions and other security or performance constraints on the DB side, not because EF is not capable of including these requirements, but because it becomes rather convoluted and unpractical to do it and i personally believe that defeats the purpose of EF, to provide you with a tool that allows for rapid development.
So, if the people involved in the project is not very comfortable writing stored procedures in SQL and the data manipulation will revolve mostly around your service without the need for very complex operations on the DB side, then EF is a suitable approach, and you can leverage the Repository pattern as well as interfaces to implement "DBContext" objects that will allow you to create a DB Agnostic DAL.
However, if you are required to implement transactions, security, extensive logging, and are more comfortable writing SQL stored procedures, Entity Framework will often prove to be a burden for you simply because it is not yet suited for advanced tasks, for example:
Imagine you have a User table, with multiple fields (Address, phone, etc) that are not always necessary for all user-related operations (such as authentication); Trying to map an entity to the results of a stored procedure that does not return any of the fields that the entity contains will result in an error, and you will either need to create different models with more or less members or return additional columns in the SP that you might not need for a particular operation, increasing the bandwith consumption unnecessarily.
Another situation is taking advantage of features such as Table Valued Parameters in SQL Server to optimize sending multiple records at once to the DB, in this case Entity Framework does not include anything that will automatically optimize operations with multiple records, so in order to use TVPs you will need to manually define that operation, much like you would if you had gone the ADO.NET route.
Eventually, you will have to weigh the considerations of your project against what your alternatives provide you; ADO.NET gives you the best performance and customization for your DB operations, it is highly scalable and allows optimizations but it takes more time to code, while EF is very straightforward and practical for objects manipulation, and though it is constantly evolving and improving, its performance and capabilities are not quite on pair with ADO.NET yet.
And regarding the drivers issue, it shouldn't weigh too much in the matter since even Oracle encourages you to use their driver instead of the default one provided by Microsoft.
I have an ADO.NET data access layer that has been written to work with multiple data providers, and am finding that I can't make some operations completely data provider independent.
As part of my application's installation functionality, it tells the DAL to create a database with a table and some data. The DAL uses the Sql command CREATE DATABASE. Now this is fine for SQL Server, but when using SQLite, the database creation step is not required, as the data source specified in the connection string is sufficient to define the database to use.
I'm currently planning on handling this by writing a 'DatabaseCreator' class with a virtual createDatabase method, which I can subclass for specific data providers, and have a factory that serves up a specific DatabaseCreator based upon the data provider being used, or the default one.
The problem with this approach is that my app won't "just work" with new data providers that aren't compatible with the default way of creating a database, but will require a new release with a new DatabaseCreator for the new data provider.
The only way around this problem I can see would be to let my app be configured with a DatabaseCreator to use in a similar way that it is configured with a data provider, i.e. the DatabaseCreator could be provided in a separate assembly.
I'm new to ADO.NET, so I'm not sure how common this type of issue is when trying to write data provider independent applications.
Is there existing support for this kind of approach in ADO.NET?
E.g. Vendor supplied or third-party existing libraries defining interfaces and/or implementations of helper classes for common operations whose implementations may be data provider dependent?
Alternatively, is there a better way to do this?
There is no defined 'standard' way, in ADO.NET, for creating a database. Some databases require a sql (CREATE DATABASE) and some embedded database need a specific API call that creates a file on the disk. So yes, you are right about your app not working out of the box.
When you would like your application to just work with a 'new database', you could get lucky if the provider works out of the box for the standard implementation (which could be executing a 'CREATE DATABASE' statement) but most of the times you will need to provide a separate implementation of the DatabaseCreator that does the database specific work.
We are making a fairly serious application that needs to remain agnostic to the DB a client wants to use. Initially we plan on supporting MySQL, Oracle & SQL Server. The tables & views are simple as are the queries (no real fancy SQL), therefore the question:
Use native DB drivers (MySQLDbConnection etc.) and encapsulate the logic of executing queries and processing results or
Use a generic OleDbConnection
Obviously option 2 involves no overhead, but I presuming the performance is not as great as with native access?
Note: This answer is relevant if you decide to use basic ADO.NET 2 functionality instead of an ORM (such as Entity Framework or NHibernate) or LINQ to SQL.
Let's assume you've got a connection string defined in your app.config:
<connectionStrings>
<add name="SomeConnection"
providerName="System.Data.SqlClient"
connectionString="..." />
</connectionStrings>
Notice the presence of the providerName attribute and its value. You could also put in a value for another DB provider, e.g. System.Data.SQLite.
(Note that non-standard providers, i.e. those that are not in the .NET Framework by default, need to be registered first, either in app.config or in the client machine's machine.config.)
Now, you can work with the specified database in a completely provider-agnostic fashion as follows:
using System.Configuration; // for ConfigurationManager
using System.Data; // for all interface types
using System.Data.Common; // for DbProviderFactories
var cs = ConfigurationManager.ConnectionStrings["SomeConnection"];
// ^^^^^^^^^^^^^^^^
var factory = DbProviderFactories.GetFactory(cs.ProviderName);
// ^^^^^^^^^^^^^^^
using (IDbConnection connection = factory.CreateConnection())
{
connection.ConnectionString = cs.ConnectionString;
// ^^^^^^^^^^^^^^^^^^^
connection.Open();
try
{
using (IDbCommand command = connection.CreateCommand())
{
... // do something with the database
}
}
finally
{
connection.Close();
}
}
Note how this code only works with interface types. The only place where you indicate a particular DB provider is through the providerName attribute value in the app.config file. (I've marked all the places where a setting from app.config is taken with ^^^s.)
Further reading:
Generic Coding with the ADO.NET 2.0 Base Classes and Factories:
similar to my answer, but goes into more detail.
ADO.NET Managed Providers and DataSet Developer Center:
includes, among other things, an index of available ADO.NET database providers.
IMHO using an ORM is a good design decision in order to have a database agnostic application. Switching database might be as easy as changing a config setting and connection string.
You don't need OleDbConnection to access nonspecific ADO.NET providers. Just use DbConnection et. al. See DbProviderFactories on MSDN for more info.
By including Oracle in that list, you've guaranteed that nothing will be simple.
Oracle uses a different prefix character (colon) for parameters, as compared to SQL Server that uses an "at" symbol.
Oracle uses a single data type (number) for long, int, short, boolean, float, and decimal; your code will have to be sure that you map these properly.
You must parameterize Oracle date and time values; if you try to use strings for dates in your SQL statements, you will go insane because of Oracle's date format. (Oracle uses a three-character month abbreviation; the format is 01-JAN-2010.)
Basic SQL functions for handling nulls can be different, particularly for null coalescing. ("NVL" versus "COALESCE") Oracle is much pickier about reserved words.
Oracle does not have native identity column support. Workarounds involve sequences, triggers, and requiring transactions just to retrieve an identity value from a new row.
In other words, your app can't be DB-agnostic. If you don't use an ORM, you will definitely want to build a data access layer that hides all these things from the rest of the application.
Voice of experience here. Just sayin'. For a common schema across SQL Server and Oracle, we've had to build most of the infrastructure of an ORM, while avoiding the aspects that can degrade performance. Interesting, but non-trivial, definitely!
LINQ is a highly regarded .NET ORM, partly because you can use it and stored procedures. Problem is, it's SQL Server only but people are working to provide similar functionality for Oracle & MySQL.
For database & query optimizations, I cringe at the idea of using an ORM. Data types, functions & overall syntax are not very portable in SQL. The most performant means of interacting with each database will be to tailor the model & queries to each one, but it means expertise, time and money. If need be, focus on one database vendor with the code setup to support vendor swap out & add support for other databases as necessary.
There's no good reason to avoid the most generic interfaces with the broadest support - OleDb and even ODBC if you're comfortable with them. Anything beyond that reduces the pool of products/languages/platforms/tools/developers you can work with. Being closest to the SQL metal, the vendor isn't going to introduce much inefficiency - certainly less than the more esoteric options. They've been around a long, long time to wring out any problems.
If you're going to add an abstraction layer (your own or someone else's), then that should be decided based on the merits of the abstractions introduced in your particular context, not just to have an abstraction layer (which is just more support unless there's an intentional benefit.)
As you can see, everyone's mileage varies. :) But in general, I think simpler is better.
Why not use the Microsoft Patterns & Practices Enterprise Library Data Access Application Block. There's minimal overhead and switching providers is a snap.
Quote:
The Data Access Application Block
takes advantage of these classes and
provides a model that further supports
encapsulation of database
type—specific features, such as
parameter discovery and type
conversions. Because of this,
applications can be ported from one
database type to another without
modifying the client code.
You can always make part of the application database agnostic by having the bulk of the application use the DAL as a bunch of interfaces. The DAL itself would then provide a concrete implementation for the target database.
This way, you get decoupling in the use of the DAL, but the benefit of performance improvements or vendor specific constructs within the DAL.