In our project, we have many small programs spread across multiple computers, each responsible for a specific tasks against the legacy database. Today, they are written in Fortran 77 and uses a very outdated framework for database access.
We are thinking of start developing in C# and investigate how we best can create a database framework that all applications can use. We can not use any existing frameworks when the old realtime database does not support SQL.
I am thinking of generate a DAL with T4 from the database definition.
The problem I see however is what happens when the database changes and DAL must be recompiled. Is it enough to copy the dll file containing the DAL to all computers, or do we have to recompile to applications?
The actual structure of the database does not change so often. However, a lot of lookup constants is changed regularly. Normally, no constants is deleted, but they can get new values or new ones can be added. And if there is any constant that is removed, the programs that are using it must anyway be rewritten.
I fear that it may become a maintenance problem and looking for a better solution.
Edit
Primary keys are not constant in the database, but are regenerated once a year. To make programs able to find the correct row in the database, lookup constants is used. In existing programs, the constants are used directly in the code in the form <TableName(RowName)>. The constants is then replaced to the current primary key value by a preprocessor. This means that all applications must be recompiled when the database is rebuilt.
It is therefore not possible to use e.g. GetByKey(int key) in BLL as the key is not constant.
I see a number of different solutions to this as listed below, which are good and which are bad? Please tell me if you see any other better solutions:
Define lookup constant in the DAL:
BLL.TableName.GetByKey (DAL.TableNameLookup.RowName)
Pros: The constant i defined in DAL and I don't have to replace BLL if the lookups is changed.
Cons: Verbose syntax
Define lookup in BLL:
BLL.TableName.RowName
Pros: Simple syntax
Cons: I have to update BLL when the lookup constants is changed.
This might be solved both with code generation (T4) and DynamicObject. Om DynamicObject is used, the constants maybe can be defined in an XML file that can be easily updated. However, it will be significantly slower.
I do not think any of these ways is good. Please help me to come up with something better.
Use the One more Layer between DAL and Application.(ex: BL-Business Layer)
Call the DAL layer methods in BL and From Application call the BL Layer. Doing this you can avoid the dependency between DAL and Application. Then when ever you do changes in database only change in DAL Layer and Replace the dll. No need to compile the application on every change in DAL.
Related
I have a VS2015 solution with a webUI project and a class lib project which acts as the domain. The class lib holds nothing more then 20 EF DB First generated classes (edmx model) and also 20 repo's which act on these classes. When I need to change the underlying db I throw away the edmx model and regenerate it. One of these classes is Domain.DbEntities.plc. My webUI references this domain lib.
After some time I added an extra project PlcCommunicator to the solution, which has a reference to the Domain lib and has methods some accepting Domain.DbEntities.plc as parameter and some returning wrapper classes which also use Domain.DbEntities.plc. My webUI project references the "PlcCommunicator" project and everything works fine.
The solution is growing larger and by the time I added more projects to it all refering and using the same Domain lib. But now I have added another project called PlcMonitoringLogger and I decided to create another smaller domain, just a subset of the main domain, which holds 5 classes which are all also just EF DB First generated edmx classes generated on the same db as the Main Domain. One of these classes is PlcMonitoringDomain.DbEntities.plc. (Note the difference with Domain.DbEntities.plc)
Now I need my PlcMonitoringLogger project to use the PLCCommunicator project. But PlcCommunicator works with Domain.DbEntities.plc and PlcMonitoringLogger only knows PlcMonitoringDomain.DbEntities.plc. So there is the problem I face.... I can change the parameters of the PlcCommunicator methods to accept plc id's instead of Domain.DbEntities.plc object's and also just return plc id's instead of Domain.DbEntities.plc objects. However, is this the right approach? Are there any pitfalls? What are the pros and cons? Another solution might be to create a base plc class, but this doesn't seem right. I want to decouple things from each other and creating base classes just doesn't feel right.
I read some stuff about bounded context's. But I can't and don't want to change al my existing projects right away to using this design pattern. Not in the last place because I have no experience in it yet and it's hard for a beginner. I think making "baby steps" to using some aspects off bounded context are the best approach, not doing total rebuilds!
So if anybody has some ideas on this topic or something useful to say please, respond!
I'm not sure what you tried to achieve by creating a subdomain, but the consequence is that they are not interchangable. So if you want to combine component which results in a domain mix-up, then you cannot do that.
IMHO, the solution is to get rid of the sub-domain, and integrate it in the current main-domain. Any project/component using the domain can without problem reference other components that use the domain as well. No multiple domains mix-up.
Some time ago, at work, we had to change our main system to be "cross-rdbms". I'm not sure if this is the correct term, but basically the system worked only with MSSQLServer but in order to acomodate a new client we had to make it possible for the system to work with both MSSQLServer and Oracle.
We don't use a ORM because of reasons. Instead, we use a custom ADO-based data access layer.
Before of this change, we rellied heavily on stored procedures, database functions, triggers, etc. A substantial amount of business logic was located on the database itself.
We decided to get rid of all stored procedures, triggers and stuff, and basically reduce to database to a mere storage layer.
To handle migrations, we created a .json file which contains a representation of our database schema: tables, columns, indexes, constraints, etc. A simple application was created to edit this file. By using it, we're able to edit existent tables and columns and add new ones.
This json file is versioned in our repository. When the application is executed, a routine reads the file, constructing a representation of the database in memory. It then reads the metadata from the actual database, compare it to the in-memory representation and generates scripts based on the differences found.
Finally, the scripts are executed, updating the database schema.
So, now comes my real problem. When a new column is added, the developer needs to:
- add a new property to the POCO class that represents a row in that table;
- edit the method which maps the table columns to the class properties, adding the new column/property mapping;
- edit the class which handles database commands, creating a new parameter referent to the new column.
When this approach was initially implemented, I thought about auto-generating and updating the POCO classes based on changes in the json file. These would keep the classes in sync with the database schema, and we wouldn't have to worry about developers forgetting to update the classes after creating new columns.
This feature wasn't implemented tough, and now I'm having serious doubts about it, mostly because I've been studying Clean Architecture/Onion Architecture and Domain Driven Design.
From a DDD perspective, everything should be about the Domain, which in turn should be tottally ignorant about its persistence.
So, my question is basically: how can I maintain my domain model and my database schema in sync, without violating DRY and without using a "database-centric" approach?
DDD puts the focus on the domain language and its representation in domain classes. DB issues are not the primary concern of DDD.
Therefore, generating domain classes from the database schema is the wrong direction if the intention is to apply DDD.
This question is more about finding a decent way to manage DB upgrades, which has little to do with DDD. Unit/integration tests for basic read/write DB operations may go a long way in assisting developers to remember editing the required files when DB columns are altered.
I have read many posts concerning the issue of having several databases and how to design a DAL efficiently in this case. In many cases, the forum suggests to apply the repository pattern, which works fine in most cases.
However, I find myself in a different situation. I have 3 different databases: Oracle, OLE DB and SQLServer. Currently, there exists a unique DAL with many different classes sending SQL queries down to a layer below to be executed in the corresponding database. They way the system works is that two of the databases are only used to read information from them, and the other one is used to either store this same information or read it later on. I have to propose a better design to the current implementation, but it seems as if a common interface for all three databases is not plausible from an architectural point of view.
Is there any design pattern that solves this situation? Should I have three different DALs? Or perhaps it is possible (and advisable) to apply the repository pattern to this problem?
Answers to your question will probably be very subjective. These are some thoughts.
You could apply command-query separation. The query side integrates directly with your data layer, bypassing any business or domain layer, and the returning entities are optimized for read and project from your databases. This layer could also be responsible to merge results from different database calls.
The command side consists of command handlers, using domain or business entities, which are mapped from your R/W database.
By doing this, the interface that you expose will be more clear and business oriented.
I'm not sure that completely abstracting out the data access layer with custom units of work and repositories is really needed: do the advantages outweigh the disadvantages? They rarely do, because you will you ever change a database technology? And if you do, this probably means a rewrite anyway. Also, if you use entity framework code first, you already have unit of work and an abstraction on top of your database; and the flexibility of using LINQ.
Bottom line - try not to over-engineer/over-abstract things; or make things super-generic.
Your core/business code should never be dependent on any contract/interface/class that is placed in the DAL layer of the application.
Accessing data is something the business/core layer of your application needs to be able to do, and this it should be able to do without any dependency of SQL statements and without having any knowledge of the underlying data access technology.
I think you need to remove any "sql" statements from the core part of the application. SQL is vendor dependent and any dependency to a specific database engine needs to be clean out of you core, and moved to the DAL where it belongs. Then you need to create interfaces that resides outside of the DAL(s) which you then create implementation classes for in one or many DAL modules/classes. Your DAL can be dependent of your core, but not the other way around.
I don't see why the repository layer can't be used in this case. When I have a database which I can only read from, I usually let the name of the repository interface indicate this, like ICompanyRepositoryRead.
If I wanted to access a database in Delphi, I could add a datamodule to a project, configure it from my mainform and then access it anywhere in the application; a reference would be stored in a global variable.
I know that in C# and other more modern OO languages global variables are frowned upon. So how can I access my database from where I need it? The biggest problem I have is the configuration: location, user, password, etc. are unknown at design time.
I now have a db-class and make a new instance when I need it, but then I would have to store those settings in some globally accessible thing, and I have simply moved the problem.
What's the standard solution?
Thanks, regards, Miel.
I always use the singleton pattern. As for configuration, look at the System.Configuration.ConfigurationManager class which allows you to read settings from your project's app.config/web.config file.
It's a bit tricky to define the absolute best practice for database access in OOP.
You've hit the nail on the head that there are a lot of factors to consider:
how are configuration parameters handled?
is the app multi-threaded? do you need database connection pools?
do you need database portability (ie: do you use different DBs in dev versus production? are you concerned about vendor lock-in with one DB? Are you distributing the app the other users who may be using a different db?)
are you concerned with securing your SQL statements, or centrally enforcing other access permissions?
is there common logic involved when performing some inserts and updates that you'd rather not duplicate everywhere a particular table is touched?
Because of this, many OOP folks gravitate to an ORM framework for anything but the simplest cases. The general idea is that your application logic shouldn't need to talk to the database directly at any point: isolate your business code from the actual persistence mechanism for as long as possible.
Instead, try to design your application so that your business logic talks to a model layer. In other words, have model objects in the system that encapsulate and describe your business data. These model objects then expose methods for obtaining and saving their state into the database, but your logic doesn't need to care about that.
For example, say you have a concept called "Person" in your system. You'd probably model this as a class with some properties. In pseudo-code:
Person:
- first_name
- last_name
Your actual code in the system is then only concerned with instantiating and using Person objects, not with obtaining DB handles or writing SQL:
p = Person.get(first_name='Joe')
p.last_name = 'Bloggs'
p.save()
In an object-oriented world, you'll find that your business logic code becomes cleaner (and shorter!), easier to maintain, and much more testable.
Of course, you're right in that this means you need to now go off and build a database back-end that translates that Person class to one or more tables in your relational database. This is where using an ORM framework comes in handy. In Python, people use Django and SQLAlchemy. I'll let others indicate what folks use in C# (I'm not a C# developer, but you did tag your question OOP, so I'm going for the generic answer here, rather than C# specific).
The point, though, is that the ORM framework puts all the DB access in a single set of classes in the code, so that the DB access, configuration and pools are handled in one place... no need to instantiate them all over the application. What you use "where you need it" is the model object.
Of course, if your app is very simple and you want just a raw DB handle, then I do recommend the dependency injection approach others have listed.
Hope that helps.
It seems to me that you need to create an appropriate object (containing the connection or similar), and pass that instance to each object requiring access (see dependency injection)
This is different from using singletons. By using this mechanism, it'll free you from the dependency on one object and (perhaps a more compelling reason in this instance) allow you to perform testing by injecting mock objects or similar in place of the originally-injected database accessor object. I would definitely shy away from the singleton mechanism in this scenario.
I actually use a repository class that takes in the db information in its constructor and have the classes that need it get it passed in. I actually use an Inversion of Control (IOC) tool to inject that values in.
You could store the user information in a flat file somewhere, then read / write to it from your db-class
This way you won't duplicate the settings in your code, but the user can still modify the settings.
SubSonic is the "Swiss Army knife" for object relational mapping, and offers the ability to execute stored procedures and return results to List. You can have it up and running within a half hour.
I'm starting with a blank slate for a layer of business/entity classes, but with an existing data source. Normally this would be cake, fire up Entity Framework, point it at the DB, and call it a day, but for the time being, I need to get the actual data from a third party vendor data source that...
Can only be accessed from a generic ODBC provider (not SQL)
Has very bad syntax, naming, and in some cases, duplicate data everywhere
Has well over 100 tables, which, when combined, would be in the neighborhood of 1,000 columns/properties of data
I only need read-only from this DB, so I don't need to support updates / inserts / deletes, just need to extract the "dirty" data into clean entities once per application run.
I'd like to isolate the database as much as possible from my nice, clean, well-named entity classes.
Is there a good way to:
Generate the initial entity classes from the DB tables? (That way I'm just mostly renaming and fixing class properties to sanitize it instead of starting from scratch.)
Map the data from the DB into my nice, clean classes without writing 1000 property sets?
Edit: The main goal here is not as much to come up with a pseudo-ORM as it is to generate as much of the existing code as possible based on what's already there, and then tweak as needed, eliminating a lot of the manual labor-intensive class writing tasks.
I like to avoid auto-generating classes from database schemas just to have more control over the structure of the entities - that and what makes sense for a database structure doesn't always make sense for a class structure. For 3rd party or legacy systems, we use an adapter pattern for our business objects - converting the old or poorly structured format - be in in a database, flat files, other components, etc. into something more suitable.
That being said, you could create views or stored procedures to represent the data in a manor more suitable to your needs than the database's current structure. This is assuming that you are allowed to touch the database.
Dump the database. I mean, redesign the schema, migrate your data and you should be good.