Develop Reference

How to represent a MySQL database schema in C#?

The title is not so accurate, but I couldn't come up with a better one.
I’m trying to write a MySQL Connector for MS‘ Forefront Identity Manager (FIM is basically a sync engine that synchronizes identities between various data sources using a meta directory). But I’m having difficulties to come up with an appropriate design.
Let’s say I want to import user data from a db into FIM’s metaverse. A user object has various attributes like firstname, lastname, address etc. In the database these attributes can be distributed between multiple tables. FIM ultimately needs these attributes to be merged into one object. So the user needs to configure the connector to tell it how the data is stored in the DB.
I was wondering what would be the “best” way to represent this configuration. Two alternatives come to (my) mind:
I could just save a select query that merges/joins the data, so that the result is a single “table” with all the desired attributes. The problem with this is that I think I would have to do some kind of parsing on this query-string to create a fim-compatible-schema out of it (which is basically the name of the object type (f.e. “person”) and a list of attributes). This schema needs to be creatable from the query-string alone without actually executing the query (I could execute some fake queries if that would simplify the process).
I could create some classes to represent the database schema, i.e. the tables and relationships. Since I’m not that experienced with MySQL (or databases at all for that matter) I’m running the risk of missing some special cases. Also it might be some kind of overkill, since the schema can be assumed as fixed once it's configured.
Does anyone have same advice on which alternative to choose and how to tackle the problems that would come with it? Or is there another – better – alternative I didn’t think of? Any advice would be greatly appreciated!
If something is not clear, please let me know.
Edit: Since there have been some questions on the use case, I'm going to elaborate a bit:
As I've said, I'm developing a Management Agent for FIM. FIM provides a so called Extensible Connectivity Management Agent, which is basically one single class implementing a few interfaces. (See this technet guide for a sample implementation).
Since I want to develop a generic agent for managing identities in a MySQL database, I don't know the database layout at compile time. When the enduser wants to use the management agent, he needs to decide, which attributes of the identities he'd like to manage. So I need to give the user some way to configure the management agent. My main question is, how to design the classes to save this configuration.
Lets look at a simple example:
Say you want to manage employee identities. To keep it simple, we have three attributes:
firstName
lastName
department
In this example case it could be f.e. just one single table with 4 columns (the attributes plus an id). But it could also be the much better design, which uses two tables, one user table and one department table, using a 1:1 relation to define the users department.
FIM requires me to consolidate these attributes in one object. It provides a class CSEntryChange which has an AttributeChanges collection member. I would then create some instances of AttributeChange (which basically contains the attribute name und it's value) and add them to the collection. So the user-editable configuration must tell the management agent how it can get the users with all defined attributes from the db and how to create and modify users in that database.
So ideally I'd have an intance of some "MySQLSchema" class (which is configured by the user up front), that could return a List<CSEntryChange> (I wouldn't actually use the CSEntryChange class for the sake of decoupling, but you should get the point) that contains all users in the db (pagination might be a requirement but I can figure that out later). In addition I'd like to able to pass it a CSEntryChange which would result in the corresponding database entries beeing updated (or created if not yet present).
I hope this clear it up a bit more :)

I think that your real question is, "How to access MySQL entities over C#?"
To begin with, I hope you are building this in as a MVC application.
I would suggest sticking to a full Microsoft stack for purposes of learning and ease of implementation.
With this in mind, you will want to create an EntityFramework MySQL data provider in the following steps:
Create a new project and and EntityFramework either through the Nuget package manager UI or package manager console by typing Install-Package EntityFramework -Version 6.0.2 (and add a reference to this project from your web project). Look half way down the page for "Configure EntityFramework to work with a MySQL database".
Install the MySQL provider for entity framework through the Nuget package manager UI or by typing Install-Package MySql.Data.Entity in the package manager console
The next step requires understanding of db configuration changes, that are nicely detailed here - Configure EntityFramework to work with a MySQL database.
You should end up with a nice class structure which will allow you to traverse your entities' navigation properties through EF.
Depending on the level of security your application requires, you may also want to create data transfer objects (DTOs) that contains only the data required for your remote calls - keeping your data calls efficient.
This is by no means a definitive guide on how to do this, but hopefully gives you a start in the right direction.
With regards to your step #1 above:
I could just save a select query that merges/joins the data, so that
the result is a single “table” with all the desired attributes. The
problem with this is that I think I would have to do some kind of
parsing on this query-string to create a fim-compatible-schema out of
it (which is basically the name of the object type (f.e. “person”) and
a list of attributes). This schema needs to be creatable from the
query-string alone without actually executing the query (I could
execute some fake queries if that would simplify the process).
I am slightly confused by this. Are you saying that you want to dynamically update your database schema based application requests?

You can use NHibernate with MySQL, and NHibernate is a full featured ORM, where C# classess maps with your MySQL tables, and the rest will be a breeze, once you get a hang of NHibernate.
A sample is here for your reference.
http://www.codeproject.com/Articles/26123/NHibernate-and-MySQL-A-simple-example

When you use the MySQL Connector/Net you can also use Entity Framework like this example from MSDN:
using (var db = new BloggingContext())
{
// Create and save a new Blog
Console.Write("Enter a name for a new Blog: ");
var name = Console.ReadLine();
var blog = new Blog { Name = name };
db.Blogs.Add(blog);
db.SaveChanges();
}

I have some experience with .NET <-> MySQL communication and I've used Entity Framework in the past for the communication - I had a lot of problems with it and performance issues and soon came to regret using it (this was 1-2 years ago, so may be they fixed it up). Of course, using an ORM framework adds a layer on top of your db communication which in my case proved to be not desired in terms of performance and flexibility.
Finally, I chose to take the following approach:
1) Create models with POCO classes as you would do with Entity Framework. Those models may or may not include relationships - it is up to your preference. I prefer to only add the relationships when I actually need them (so some objects may have their db relationships in the POCO's and some may not). I chose this because it lowers the complexities of when to pre-load the relationships and when not. Basically, if you don't need it - don't add it.
2) Create DAL layer (for example, using the repository pattern) that accepts and works with those objects and fires direct queries to MySQL. No EF required for this - you just need to install the Connector/NET for MySQL and you are ready to go.
A quick example of this would be the following (note: example is of the top of my head and it is just to illustrate the classes. I would use command parameters as well to prevent injection and so on):
public class Person{
public string Name {get;set;}
}
public interface IPersonRepository{
void AddPerson(Person p);
}
public class PersonRepository{
public void AddPerson(Person p){
using(var connection = new MySqlConnection("some connection string"){
connection.Open();
var command = new MySqlCommand(connection);
command.Text = string.Format("insert into Person (Name) values ({0})", p.Name)l
command.ExecuteNonQuery();
}
}
}
The benefits of this approach for me are:
Performance - my application need to insert large amounts of data int MySQL. Entity Framework could not cope with this. If your application doesn't handle a lot of data you might be alright with EF.
Flexibility - writing my own queries allows me to have better control over the communication. You can choose, for example, to use bulk inserts in MySQL (from file - really powerful and fast when you need to handle large amounts of data) for which you will need to bypass Entity Framework. I also found out that EF generates some funky queries
The main drawback is, of course, more work - you will get some things for "free" with the Entity Framework.
So, I can recommend the following:
Consider the amounts of data that you need to handle and make a small exercise application with those amounts. How does EF (or any other ORM) handle it? What about direct queries to the database? That will give you a somewhat accurate idea of how the communication will perform.
Consider how much time you have for building this application - if you are looking for a quick solution and are willing to sacrifice a bit of performance - go for EF or another ORM framework. If you have more time on your hands and would like to make a flexible solution - go for direct queries to the database.
Good luck!

Use Entity Framework Code First.
http://msdn.microsoft.com/en-us/data/jj193542.aspx
It is still a lot of work, but I think this is the quickest approach.
Create a C# classes according to the user and create the DB schema from those classes.

How to create a C# class whose attributes are rows in a database table with ADO.NET?

Is it possible?
Please note I am not using LINQ nor Entity Framework.

You could also check out Dapper-Dot-Net - a very lightweight and very capable "micro ORM" which - incidentally - is used to run this site here.
It's quite fast, a single *.cs file, works with your usual T-SQL commands and returns objects - works like a charm, it's very fast, very easy to understand, no big overhead - just use it and enjoy!

My personal favorite is done using the dynamic object featured in .NET4 via Rob Conery's Massive library. Like Dapper-Dot-Net it is small.

By going old school you can use Datasets to create strongly typed data table classes that mirror your database entirely right down to the relationships. It's a precursor to LINQ/EF that auto-generates a lot of bloated code but they're very handy for maintaining your field names, data types, data constraints and performing easily configured rapid updates.
http://msdn.microsoft.com/en-us/library/esbykkzb(v=VS.100).aspx

Database Design In SQL Server or C#?

Should a database be designed on SQL Server or C#?
I always thought it was more appropriate to design it on SQL Server, but recently I started reading a book (Pro ASP.NET MVC Framework) which, to my understanding, basically says that it's probably a better idea to write it in C# since you will be accessing the model through C#, which does make sense.
I was wondering what everyone else's opinion on this matter was...
I mean, for example, do you consider "correct" having a table that specifies constants (like an AccessLevel table that is always supposed to contain
1 Everyone
2 Developers
3 Administrators
4 Supervisors
5 Restricted
Wouldn't it be more robust and streamlined to just have an enum for that same purpose?

A database schema should be designed on paper or with an ERD tool.
It should be implemented in the database.
Are you thinking about ORMs like Entity Framework that let you use code to generate the database?
Personally, I would rather think through my design on paper before committing it to a DB myself. I would be happy to use an ORM or class generator from this DB later on.

Before VS.NET 2010 I was using SQL Server Management Studio to design my databases, now I am using EF 4.0 designer, for me it's the best way to go.

If your problem domain is complex or its complexity grows as the system evolves you'll soon discover you need some meta data to make life easier. C# can be a good choice as a host language for such stuff as you can utilize its type-system to enforce some invariants (like char-columns length, null/not null restrictions or check-constraints; you can declared it as consts, enums, etc). Unfortunately i don't know utilities (sqlmetal.exe can export some meta but only as xml) that can do it out of the box, although some CASE tools probably can be customized. I'd go for some custom-made generator to produce the db schema from C# (just a few hours work comparing to learning, for example, customization options offered by Sybase PowerDesigner).

ORMs have their place, that place is NOT database design. There are many considerations in designing a database that need to be thought through not automatically generated no matter how appealing the idea of not thinking about design might be. There are often many things that need to be considered that have nothing to do with the application, things like data integrity, reporting, audit tables and data imports. Using an ORM to create a database that looks like an object model may not be the best design for performance and may not have the the things you really need in terms of data integrity. Remember even if you think nothing except the application will touch the database ever, this is not true. At some point the data base will need to have someone do a major data revision (to fix a problem) that is done directly on the database not through the application. At somepoint you are going to need need to import a million records from some other company you just bought and are goping to need an ETL process outside teh application. Putting all your hopes and dreams for the database (as well as your data integrity rules) is short-sighted.

pluggable data store architectures

I have a pluggable system management tool. The architecture of this kind of thing is well understood (interfaces, publish/ subscribe, ....). How about the data store though. What do people do?
I need plugins to be able to add new entities, extend existing entities, establish new relationships, etc.
My thoughts (SQL), not necessarily well thought out
each plugin simply extends the schema when they are installed. In the old days changing the schema was a big no-no; now databases are very relaxed about this
plugins have their own tables. If 2 of them have an entity (say) person, then there are 2 tables p1_person and p2_person
plugins have their own database
invent some sort of flexible scheme where the tables are softly typed. Maybe many attributes packed into a single attribute. The ultimate is to have one big table called data, with key of table name & column name and a single data value.
Not SQL
object DB. I have no experience with these. Anybody care to pass on experience. db4o for example. Can I change the 'schema' of objects as the app evolves
NO-SQL
this is 'where its at' at the moment. Most of these seem to be aimed slightly differently than my needs. Anybody want to pass on experience with these
Apologies for the open ended question

My suggestion is go read about the entity framework
a lot of the situations you are describing can be solved (very elegantly) using table inheritance.
Your idea of one big table called data makes the hamsters in my computer cry ;)
The general trend is away from weakly typed schemas because they cannot be debugged at compile time. What you get from something like entity framework is a strongly typed extenislbe schema that you can code against using linq.
Object databases:
like you i havent played with them massivley - however the time when i was considering them was a time when there was no good ORM for .net and writing ado.net code was slowly killing me.
as for NO-SQL these are databases that meet a performance need. SQL performs badly in situations here there are lots of small writes occuring. I say badly tounge in cheek - it performs very well but when you scale to millions of concurrent users everything changes. My understanding of no sql is that it is a non rationalised format designed for lots of small fast writes and reads. The scale of sites that use these is usually very large.
OK - in response
I am currently lucky enough to be on a green field project so i am using EF to generate my schema.
On non greenfield projects I use sql scripts to update my table structures. As for implementing table inheritance in sql its very easy once you know the concept, its essentially a one to many relationship with a constraint that it will only ever be 0-1.
I wouldn't write .net code that updates the database structure ... that sounds like a disaster waiting to happen to me.
Beginning to think i have misunderstood what you are looking for. I find databases to be second nature as I have spent so long with them.
I haven't found a replacement for being meticulous about script management.

C# and MySQL - Gentle Framework alternatives

I'm playing around at the start of a personal project in C# and MySQL.
I am familiar with the use of the Gentle Framework (using MyGeneration to generate the classes, based on the data model). Here's what I like about Gentle;
Simple-to-use [class].Retrieve(id) / [object].Persist() semantics with strong-typing of fields;
I start with the DB data model, and choose when to generate new code files;
MyGeneration allows for some 'manual code sections' which are kept across generations...
...and partial classes allow me to add permanent code in parallel files, e.g. simple read-only properties (like 'FullName' from FirstName and Surname members for a Person object) - or I could use inheritance;
I find it a tolerable and quick way to create a DAL, and add certain Business-Object-Layer-like facilities to it.
Unfortunately, to query efficiently, I end up using queries / SqlCommands a fair bit, and relies on weakly typed references to column names etc., and appears to risk sidestepping the object broker and therefore caching advantages. In any event, Gentle is no longer being developed, and it seems like a good time to consider alternatives.
So, what should I consider?
Generation of strongly-typed ADO Datasets is possible, but it seems like it will be difficult to add to it (e.g. that 'FullName' virtual column) in a way that will persist after updates to the table structure with regeneration of the dataset.
NHibernate seems to have lots of fans... but my first looks into it seem to suggest that the XML data definition is king, not the existing data-model in the DB. It also looks quite heavy on dependencies;
The SubSonic demo appears to suggest it generates files, and in the demo of WebAppProjects, looks like it might generate files in a way that I could add to, or inherit from;
The MySql Connector.Net tools appear not to support the dataset generation for Linq (e.g. via drag-and-drop), and I suspect that this is a key need for strongly-typed data access.
Your thoughts will be gratefully appreciated! Thank you in advance...

I had some experience with Gentle and I do have to admit that it was pretty inefficient with queries. I would suggest looking into NHibernate, since it has a rich community. It is true that XML definitions are preferred, but there are ways of doing the mappings using class-level attributes.
SubSonic (especially the 3.0 version) looks very promising with its use of T4 templates. That should give you more control over code generation. It can do LINQ too.
Don't invest in LINQ-to-SQL, since the rumors are that is going to be discontinued.

Assuming that the .Net 3.5 Framework is an option for being used, then you can take a look at Microsoft's Entity Framework (released with .Net 3.5 Service Pack 1).
The Entity Framework allows the generation of DAL classes based on your database schema, but the maintenance of these classes are hidden behind an XML file that can quickly and easily be updated to account for schema changes by a simple command from the Visual Studio IDE.
I am working on a project where we use the Entity Framework with MySQL with few problems.
The main disadvantage to this option is that the official .Net connector provided by MySQL does not yet support the Entity Framework - there is a paid alternative known as MyDirect.Net

link textI would go for Subsonic, mature DAL generator and improves productivity by great margin.
We have used it with both MySQL and SQL Server - no headaches. Generates classes for Tables, Stored procedures, column names. So every time we find ourselves doing Somthing Dot Intellisense Move Arrow keys and semicolon.
Any time your schema changes, you can regenerate those classes and you are home. Also, you can extend them by creating partial classes.
It supports almost all of the SQL Semantics - Joins, loading Collection by primary key, adding WHERE clause, Order by, Count, Top, Calling stored procedures, views and so on and Intuitive syntax is big plus.
To give you some glimpse- For Books table[BookID-PK, title, AuthorID], It generates several types of methods.
Insert method which takes Title, AuthorID
Nullable columns are optional
parameters a.k.a C# Nullable type ?
Update method wich takes BookID, AuthorID, Title
Load Book by Primary key (Useful when displaying detail page)
BookCollection and Book Entities, Just call BookCollection.Load and you have list of books ready to bind to any databound control
Here's quick link.
Thanks,
Maulik Modi

Thanks to both Filip and Snorkpete for your suggestions - your comments and links proved helpful.
I will probably try SubSonic first; it looks like something I will understand and be able to get going with quickly (today should answer that), and I was surprised to see that it is indirectly supported by MS as they employ the guy who writes it. T4 also looks very interesting.
The Entity Relationship Model also looks interesting, and the link to MyDirect may be helpful in the future. The only down side here is one of expectation; MS have screwed-up their approach in the past by making them easy to create the initial design with drag-and-drop, then much harder later to modify or keep up-to-date.
Anyway, thank you both again, and I'll try to keep this question updated.
Nij

I use a bit of SQL to generate strongly typed objects out of tables, it's based on one built by Cade Bryant, but I've made some tweaks. The code it generates is not 100% compilable but it saves a lot of boiler plate work and the gaps are easy to fill (i would make all the properties fully fledged properties if i were you, or bear the wrath of jon skeet!)
http://NotifyURL.com/sql