Develop Reference

Is it possible for Lucene to monitor a Sql Table and keep itself updated?

I am trying to understand some basics of Lucene, the full text search engine. More specifically I am looking at Lucene.Net.
Today I have an old legacy .NET 4.8 web app. Some is MVC, but the newer parts follow a pretty nice API first pattern. The app holds a lot of records (app half a million) with tons of different fields. The search functionality there is outdated to say the least. It is a ton of old Linq2SQL queries that fan out in like queries.
I would like to introduce a new and better way to search records, so I started looking at Lucene.Net. But I am trying to understand one key concept, and I can't seem to find the answer anywhere, and I think it might be because it cannot be done, but I would like to make sure.
Is it possible to set up Lucene to monitor a SQL table or view so I don't have to maintain the Lucene index from within my code. The code of this app does not lend itself to easily keeping a Lucene index updated when things are added, changed or deleted. But the database is good source of truth. I can live with a small delay on having the index up to date. But basically I would like define for each business model what fields are part of the index and what the id is, and then be able to query with that index from the C# server side code of my Web App.
Is such a scenario even possible or am I asking too much?

It's totally possible, but not out of the box. You have to implement it if you want it. Fundamentally you need to implement three things.
A way to know every time a piece of relevant data in the sql database changes
A place to capture information about that change, call it a change log.
A routine that reads the change log, applies those changes to the
LuceneNet index and than marks the record in the change log has processed.
There are of course lots of different ways to handle each of these.
This SO answer Lucene.Net index updates, when a manual change is done in SQL Database provides more details on one way this can be accomplished.

Building C# and SQL Server database dependant application

Let's say I need to create an application like a books library management system for example that has a front-end like Windows form/WPF and it stores information to database. How to approach making such a solution.
Do we need to create a database first with all tables in C# or is the other way.. SQL to C#?
How do people generally do this? Can someone point me to a sample free project or a book that does this to reinforce my understanding.

Both ways are possible. People usually only focus on one side and use a tool for the other.
Database first approach: Codesmith is an exemple of tool that'll generate C# files after you created the database.
Code first approach: Entity Framework is (an exemple) for the other way around, you write your model in C# and it will generate the database accordingly.
Now pointing you at one or the other would be a bit subjective and also not really match Stack Overflow spirit.

SQL table / view designer in c#

I am developing an application that requires "some" customization by the enduser afecting the database design (beyond the parametrization).
Now a days this application supports to work with new columns for the existing tables or even new tables in runtime. But the enduser is not capable to alter the tables, and all the design work must be done using Microsoft SQL Server Magament Studio.
My question is: there is any kind of control (or tool) which implements this functionality? I would like to have it embeded in my application, but if it is a external tool wouldn't be so bad.
What I want to have is some tool that let the user define a repository of columns (name, type, size) for example:
CustomerCode, BIGINT, n/a
CustomerName, Varchar, 50
And then create or alter a table or view by adding these predefined columns. And underneath should execute the needed SQL script or maybe using SMO (SQL Management Objects).
Updated
Currently it's the application works with WinForms, but any WPF / ASP solution would be appreciated.
And referring to and enduser I mean a app administrator but not with programing skills.
Purpouse
The purpose is to be able to extend and customize the functionality from the own apllication without having to use SQL Server Managment Studio.
Imagine that you have an ERP, as a user you want to inform in your customers table something that the application was not orginally intended and even has no free fields for it, for example: GPS location, logo, CEO's photo...
Of course I can suggest to some advanced users to install SSMS Express, but that will give them too much freedom. I also think it is a IT tool. What I wanted is to develop the capability to modify the application from the own framework of the application.
If it can be done from the own application, some controls can be performed:
Check that a table field called X will be always defined with the same type,
Disable the capability to modify the non-customizable fields (application fields),
...

Alex, I think you should try to give the user the flexibility to add new information to the database without really change the design of the database.
You can take a look at EAV Model (entity-attribute-value). This give you the flexibilty to add any new attribute to customer without change the design of customer table (e.g.).
One good example is Magento, they did a great job with EAV (entity-attribute-value) but you must know that this design model will hit your performance a little (or a lot depends how you implement it).

It seems that this kind of control does not exists, so I will scratch it from 0.
#BrunoCosta's idea of using an EAV model to extend the standard application is a good practice. But I pretend to have a tool to modify the standard, and have a tool used for the final user and also for the developer. A pseudo SQL Server Studio + Visual Studio embedded in the a application.
Once I develop it, I will to post it in CodeProject.

I think that the control you are looking for may be a little too unique. However, I don't think it would be too difficult create this yourself using other controls/libraries.
I am just finishing up a similar WPF project.
I found that the DataGrid class worked well for representing a table. You can add/remove columns programmatically. Although you would be generating the SQL script yourself.

How to transfer data (Entities) from one database to another

I have a system (using Entity Framework) that is deployed in various production systems and also on a quality control system. My problem is that data entry is often done only on one of those occurrences of my system (different databases).
I want to find the best way to transfer my data from one database to another database. Ids can change, as long as the relations between my objects are maintained. 98% of my data in in DB, some of it is external files, I can manage those separately, manually.
Currently we use a xml structure as a transition file. The file is then imported in the destination system, and code manually imports the entities and re-creates the data.
I'm looking for a more generic way to do this, with less code. Since all my data in stored in Entities couldn't I simply create a big List and throw all my objects in there, then serialize that in some matter into an external file and finally generically import all the entities in there in my destination system? (I'll probably have to be careful in maintaining relation ids, but should be ok...)
Anyways I'm wondering if anyone would have smart approaches, I'm pretty sure I,m not the first with a similar problem.
Thanks!

You need to get some process around this. If all environments contain the same data (unlikely) you can replicate. It is the most automatic. But a QA environ should not update production, so you have to really think this through.
If semi-automated is okay, there are tools out there you can use from a variety of vendors. I use Red Gate tools, personally, but others are also fine.
Can you set up a more automated push with EF? Sure, but the amount of time you spend is really not worth it.

In my opinion you can check some of the following approaches:
1) Use Sql Compare or Sql Data Compare. Those tools are from Red Gate and can be found here
2) Regular backups and restores of the databases. You could, if it is an option regularly backup your most up-to-date database and restore it on the destination systems. I have no experience in automatizing this but here is a link to do that through .net.
3) You could always give it a go creating a version control system of your own. I would picture one such system selecting all records from a certain table (or all of them), deleting all records in the target database and inserting them. This seems pretty complex though, as you have to worry about relationships, data dependencies, etc.
Hope this helps in some way.
Regards

If you for some reason will not be satisfied with existing tools may be you'll want take a look at the Sync Framework and implement this functionality yourself for your very particular data bases.

Given what you described, pushing data from One SQL Server to another for demo purposes, you should consider SQL Server Integration Services.
If you're got a simple scenario where you just move the data and objects from DB to the next you can use their built-in Wizards. If you need to do custom stuff you can build complex workflows using C# and SQL (tools you already know). Note: most of what you're going to want comes with the standard edition so if you're using express this is less interesting.
The story for Red Gate products is more compelling when you don't have SQL Server (So you have to go out and buy something) and if you are interested in finding out what the changes are between DB's (like viewing code changes in a .cs file in a source control product)

Database Design In SQL Server or C#?

Should a database be designed on SQL Server or C#?
I always thought it was more appropriate to design it on SQL Server, but recently I started reading a book (Pro ASP.NET MVC Framework) which, to my understanding, basically says that it's probably a better idea to write it in C# since you will be accessing the model through C#, which does make sense.
I was wondering what everyone else's opinion on this matter was...
I mean, for example, do you consider "correct" having a table that specifies constants (like an AccessLevel table that is always supposed to contain
1 Everyone
2 Developers
3 Administrators
4 Supervisors
5 Restricted
Wouldn't it be more robust and streamlined to just have an enum for that same purpose?

A database schema should be designed on paper or with an ERD tool.
It should be implemented in the database.
Are you thinking about ORMs like Entity Framework that let you use code to generate the database?
Personally, I would rather think through my design on paper before committing it to a DB myself. I would be happy to use an ORM or class generator from this DB later on.

Before VS.NET 2010 I was using SQL Server Management Studio to design my databases, now I am using EF 4.0 designer, for me it's the best way to go.

If your problem domain is complex or its complexity grows as the system evolves you'll soon discover you need some meta data to make life easier. C# can be a good choice as a host language for such stuff as you can utilize its type-system to enforce some invariants (like char-columns length, null/not null restrictions or check-constraints; you can declared it as consts, enums, etc). Unfortunately i don't know utilities (sqlmetal.exe can export some meta but only as xml) that can do it out of the box, although some CASE tools probably can be customized. I'd go for some custom-made generator to produce the db schema from C# (just a few hours work comparing to learning, for example, customization options offered by Sybase PowerDesigner).

ORMs have their place, that place is NOT database design. There are many considerations in designing a database that need to be thought through not automatically generated no matter how appealing the idea of not thinking about design might be. There are often many things that need to be considered that have nothing to do with the application, things like data integrity, reporting, audit tables and data imports. Using an ORM to create a database that looks like an object model may not be the best design for performance and may not have the the things you really need in terms of data integrity. Remember even if you think nothing except the application will touch the database ever, this is not true. At some point the data base will need to have someone do a major data revision (to fix a problem) that is done directly on the database not through the application. At somepoint you are going to need need to import a million records from some other company you just bought and are goping to need an ETL process outside teh application. Putting all your hopes and dreams for the database (as well as your data integrity rules) is short-sighted.