Develop Reference

Best practice for data version control in sql and c#

In this question i want to figureout, what is the best practice to control versions of data in sql. We are useing a relational database (Sybase SAP Sql Anywhere). The problem is, we don't know in which layer of our software we should implement a version control system. We want to write a generic system, so that version control is available for all types of data with a small amout of work for every type (Types: Contacts, Appointments, ...).
Here are the options we figured out:
1. Using an entity framework and calculating the difference of two models. Then saving the difference to the database
2. Using triggers and comapre old and new data and save them in separate table
3. Using procedures which proof for changes and save them also in a separate table
I know it's a very general question, but maybe some one has a good idea and solution for our problem.
Edit
Important: I want to create versions of the data itself, not of the sql schema or some sql code.
EDIT2
Lets use the following simple example. I have a tiny contact table (not our real contact table):
CREATE TABLE Contact
(
"GUID" Uniqueidentifier NOT NULL UNIQUE,
"ContactId" BIGINT NOT NULL Identity(1,1),
"Version" INTEGER NOT NULL,
"FirstName" VARCHAR(100),
"LastName" VARCHAR(200),
"Address" VARCHAR(400),
PRIMARY KEY (ContactId, Version)
);
No, every time some one made changes to the contact object, i want to save a new version of it. But im am looking for a general solution. This must be implemented for every type.
Thank you!

As someone who live and breathe database source control (part of amazing team at DBmaestro), I can recommend on combination of 2 methods, depending on how you run the delta.
Using triggers you should save all information you need for the deployment, if it by using slow change dimension or entire table content
Using procedure that analyze difference and knows to generate the relevant delta script

we have the same issue and try to solve it by storing a version and a branch id with every entity we want to follow.
In a different table we store the versions with their predecessor version id, so we can trace where branches meet each other.
Seperatly we have an audit trace with the version number.
I wonder if this has the same elements you need(ed) and whether you advanced since your question and your last edit.
Thanks for the suggestion to combine the unique id and the version number

How can I retrieve new rows inserted in a table from a .NET application?

I'm working with C# and Microsoft SQL Server Express 2008. The table will be populated by a remote hardware so i don't have control over SQL command for the insert. Can I set SQL server to generate an event on insert catchable in C# (or other .NET) application?

There are many ways to do that if i would you then i will do something like....
Write insert Trigger on that table and in that trigger call exe
For example :
declare #sqlcmd varchar(200)
SET #SQLCmd = 'c:\dba\sampl_2.exe'
EXEC master..xp_cmdshell #SQLCmd , no_output
and in that exe you can handle whatever you want...

Probably you are looking for: SqlDependency class.
But as far as I remember SQL Server Express does not support SqlDependency features.
If so, I would create one more column, something like "IsProcessed" and once the app processes the row set it to true or something. By using this approach you can query the table using where clause: where IsProcessed is null.

There are several options to do something like this. It's easy to set up an insert trigger in SQL that effectively fires an event during which you can carry out any number of tasks: send a message, write to another table, write to a log. etc.
For me, the question is how best to you get your C# program to "listen" for this trigger event.
One option might be to set up a little WCF program as as listener that responds to messages sent by SQL server when the insert trigger fires. Here's a link to a CodeProject piece about doing something like that.
http://www.codeproject.com/Articles/21149/Invoking-a-WCF-Service-from-a-CLR-Trigger

It's interesting how this kind of question emerges on SO from time to time. And there is still no explicit unique answer that can be given in all situations.
Actually, the question asks on how should a multiple-insert-event be captured in SQL server for a particular table. I hope it is a particular table, because in SQL Server itself there isn't a method to listen DML changes to all tables (something like * from tables) at once. If you want, you can create N triggers (it could be done e.g. using dynamic SQL) on N tables and listen to these, but what if a new table gets added?
Let's think we have a specific table MyTable where we would like to listen for INSERT, UPDATE and DELETE events. One way to do it is implementing a trigger (speciffically AFTER trigger, since INSTEAD OF triggers are not made for this case). Another option is a query notification. But which one do you really need?
Query notification implies that you have to code .NET application. Trigger implies all that is trigger-related: you can have TSQL code, you can even call a web service from your database using CLR trigger. But - what do you need?
So, if you really need to react in the way to just write a log record in a table, use plain old trigger. Otherwise you should think: why do you need really need C# application here?
For example, say you need to catch a delete statement and serialize all deleted rows in XML format to the file system in a file. I would use CLR trigger in EXTERNAL_ACCESS mode to be able to create file and save information to disk.

How can I keep track of changes to a Database?

I am creating a website that will be used by an accounting dept. to track budget expenditures by different projects.
I am using SQL Server 2008 R2 for the database and ASP.net C# MVC 3 for the website.
What my boss has asked me to do is every time any user updates or creates a project, we need to log that change into a new table called Mapping_log. It should record the whole Mapping row being saved or created, and additionally the user and the datestamp. The notes field will now be mandatory, and the note should be saved to the Mapping_log.
Now when editing the PA, the Notes field will always be empty and below it, it should have a list of the older notes organized by date. I have been looking into maybe using Nlog and Log4net but I have not been able to find any good tutorials for a situation like mine. It seems that those modules are mostly used for error logging, which although important is not exactly what I am try to do at the moment.
I need some direction... does anyone have any advice or tutorials that I could use to learn how I can implement a process that will keep track of changes made to the data by users of the site.
Thanks for your help/advice!

You can consider two new features that SQL Server 2008 introduced: Change Tracking and Change Data Capture.
You could use that and avoid your custom Mapping_log table.
But if you need to apply a more complex -business- rule, perhaps it will better doing that in the application layer, rather than purely in the database.
Regards.

I would just create two triggers - one for the update, one for the insert.
These triggers would look something like this - assuming you also want to log the operation (insert vs. update) in your Mapping_Log table:
CREATE TRIGGER trg_Mapping_Insert
ON dbo.Mapping
AFTER INSERT
AS
INSERT INTO dbo.Mapping_Log(col1, col2, ..., colN, User, DateStamp, Operation)
SELECT
col1, col2, ..., colN, SUSER_NAME(), GETDATE(), 'INSERT'
FROM
Inserted
(your UPDATE trigger would be very similar - just replace "insert" by "update" wherever it appears)
This is done "behind the scenes" for you - once in place, you don't have to do anything anymore to have these operations "logged" to your Mapping_Log table.

Building dynamic SQL queries in .net 2?

I could be re-inventing the wheel - but..
I need to allow a user to be able to build 'customer reports' from our database - which will be from a GUI.
They can't have access to SQL just a list of Tables (Data groups) and columns within those groups.
They also have the ability to create Where clauses (criteria).
I've looked around Google - but nothing cropped up.
Any ideas?

well my recommendation, Developer express have some amazing end user criteria builder, you can use theirs.
there are other controls to create end users criteria , like
http://devtools.korzh.com/query-builder-net/
I hope that help you
both controls above abstract the data acess layer so your end users wont have access to send a direct query to the database. The controls only build the criteria and its your work to send the query to the database.

As a precurser to my answer, there are a number of expensive products out there like Izenda (www.izenda.com) that will do this very elegantly...
If you want to roll your own (and to speak to your question) you can fake this pretty quickly (and yes, this does not scale well to more than about 4 joins) like this:
Create a join statement that encompasses all of your tables that you want to use.
Create a dictionary of all available fields you want to expose to the user such as this: Dictionary = Dictionary<[Pretty displayable name], [fully qualified Sql field name]>
Let the user build a select list of fields they want to see and conditions they want add from the dictionary above and use the dictionary value to concat the sql string together that is necessary to return their results.
(I'm skipping quite a bit of validation work about making sure that the user doesn't try to mis-type the condition and such, but essentially point of this is that for a small collection of tables you can create a static "from" statement and then safely tack on the concat'ed "select" and "where" that the user builds)
Note that I've worked on some systems that actually stored the relationships of the table and compiled the most efficient "from" statement possible... that is not a huge stretch from this answer, it's just a bit more work.

I strongly recommend going with an existing product like Crystal Reports, Sql Server Report Builder, or Infomaker. It's just so easy to get something that seems to work, but leaves you open for an sql injection attack.
If you do go ahead, I recommend using a separate sql connection for these reports. This connection should have a user account that only has read privileges anywhere in the database.

Thanks for the answers! We ended up doing this ourselves through a collection of views!
For instances:
Sales View
Customer View
The views already take care of most of the joining between tables and return as much joined data as they can.
The user then selects what columns they could like to see from each view and we do the join between the views at code level.
The resulting statement is very small as the views take most of the work out of it.

Best way to implement form editing and auditing all edits?

I'm starting a new project which will need to allow edits on forms but to keep track of the original and who did what edits and where (p.s. I wouldn't be able to use any extra software other than visual studio 2010 and Microsoft SQL Server Management Studio so no point suggesting any addition software, this is purely a code or table design minded question) .
I'm a perfectionist and I know some possible routes to achieve this will prob change my overall project design but I'm not sure if the ideas I have on how to implement this are best so I like to hear others opinions on below ideas and your own ideas on the quickest most effective way to implement above problem.
Ideas:-
I'd set it up so that when they edit it would display all existing ranges of data from textboxs to radiobuttons and even some drop downs and the value which they had and then on submit it would copy the original record via the Id into a achieve table, create the new record and then delete the original from the main table.
I figure some way to add X amount of comments to any section of the form and each would have a timestamp and username from win auth recorded at the bottom.
Edit - My intention was to get a variety of solutions but I suppose once I'm able to start on the editing section of this project if the single solution given works then I'll mark that correct.

I'm not sure whether this is what you are looking for but I have the need to log all changes to data (for audit reasons) and the way I have implemented this is to create a new 'History' table in SQL Server that will store the record ID, username of person who changed it, whether they added/modified/deleted something and when this happened etc.
In the code to add/edit/delete things in my database I always call ObjectContext.SaveChanges (I use Entity Framework 4) so what I have implemented is an extension to this method that uses various parts of the ObjectStateManager to get the information required about the entity that has changed and inserts the details into the History table. You then just need to query this table in the database to display details of what has changed.