Develop Reference

Adding Entity Objects to LINQ-to-SQL Data Context at Runtime - SQL, C#(WPF)

I've hit a wall when it comes to adding a new entity object (a regular SQL table) to the Data Context using LINQ-to-SQL. This isn't regarding the drag-and-drop method that is cited regularly across many other threads. This method has worked repeatedly without issue.
The end goal is relatively simple. I need to find a way to add a table that gets created during runtime via stored procedure to the current Data Context of the LINQ-to-SQL dbml file. I'll then need to be able to use the regular LINQ query methods/extension methods (InsertOnSubmit(), DeleteOnSubmit(), Where(), Contains(), FirstOrDefault(), etc...) on this new table object through the existing Data Context. Essentially, I need to find a way to procedurally create the code that would otherwise be automatically generated when you do use the drag-and-drop method during development (when the application isn't running), but have it generate this same code while the application is running via command and/or event trigger.
More Detail
There's one table that gets used a lot and, over the course of an entire year, collects many thousands of rows. Each row contains a timestamp and this table needs to be divided into multiple tables based on the year that the row was added.
Current Solution (using one table)
Single table with tens of thousands of rows which are constantly queried against.
Table is added to Data Context during development using drag-and-drop, so there are no additional coding issues
Significant performance decrease over time
Goals (using multiple tables)
(Complete) While the application is running, use C# code to check if a table for the current year already exists. If it does, no action is taken. If not, a new table gets created using a stored procedure with the current year as a prefix on the table name (2017_TableName, 2018_TableName, 2019_TableName, and so on...).
(Incomplete) While the application is still running, add the newly created table to the active LINQ-to-SQL Data Context (the same code that would otherwise be added using drag-and-drop during development).
(Incomplete) Run regular LINQ queries against the newly added table.
Final Thoughts
Other than the above, my only other concern is how to write the C# code that references a table that may or may not already exist. Is it possible to use a variable in place of the standard 'DB_DataContext.2019_TableName' methodology in order to actually get the table's data into a UI control? Is there a way to simply create an Enumerable of all the tables where the name is prefixed with a year and then select the most current table?
From what I've read so far, the most likely solution seems to involve the use of a SQL add-on like SQLMetal or Huagati which (based solely from what I've read) will generate the code I need during runtime and update the corresponding dbml file. I have no experience using these types of add-ons, so any additional insight into these would be appreciated.
Lastly, I've seen some references to LINQ-to-Entities and/or LINQ-to-Objects. Would these be the components I'm looking for?
Thanks for reading through a rather lengthy first post. Any comments/criticisms are welcome.

The simplest way to achieve what you want is to redirect in SQL Server, and leave your client code alone. At design-time create your L2S Data Context, or EF DbContex referencing a database with only a single table. Then at run-time substitue a view or synonym for that table that points to the "current year" table.
HOWEVER this should not be necessary in the first place. SQL Server supports partitioning, so you can store all the data in a physically separate data structures, but have a single logical table. And SQL Server supports columnstore tables, which can compress and store many millions of rows with excellent performance.

One Dataset for each DataTable or one Dataset for all DataTables?

I am writing a C# application that uses Local DataSets.
I have a DataSet for Customer Information, and a DataSet for Stocked Items.
In regards to perfomance, and file size, would it be advisable to use just one DataSet with the Customer Infomation and the Stocked Items DataTables, or separate DataSets for each DataTable?
I intend to use data to put into a report (using report viewer).

The DataSet is built for, well just that, a set of data. It's built to allow you to create relationships between tables, provide cascading, and constraints. You should use data sets to group tables together when you are working with more than one table at a time. If you're not working with more than one table, get rid of the DataSet and just use a DataTable.
Now, if you're trying to cache all of the data locally, then that would make sense for all the tables to be in one DataSet where the data can be related and operated on together. Well, that is if you're operating on those tables together.
UPDATE: based on your comment about your intentions, it makes sense for all of the related data to exist in a single DataSet. The report is going to need to be able to know what rows are related and in what ways to handle grouping and such.

How to interrupt in fetching data from DB

We use LINQ to SQL in our project. One of the tables is "Users" used in every action in the project.
Recently we were said to add "IsDeleted" column to the table and consider that column in every data fetching in LINQ to SQL queries.
We wouldn't want to add "WHERE IsDeleted = Fasle" to all queries.
Is it possible "to interrupt" to LINQ after the data was fetched but before sending further to code in the project?

This can be solved by C# but it would really be the wrong tool for the job.
Create a view in the database that includes this statement and only work with the view from now on. You can even enforce this by not granting privileges on the table any more.

Using DataAdapter .Update to insert/upd rows in dataset (not based on DBs PK) problem

I have a batch process that reads data from multiple tables into a dataset based on a common key. I then build a second dataset of the destination data querying on the same key.
At this point I have two Datasets that are structurally identical (from a table/column layout perspective). I then have a process that adds any row that exists in source to the destination dataset. In addition, the process will attempt to update certain columns based on the common key as well.
The problem seems to come in when the DataAdapter.UPDATE command is called with existing rows that it needs to update.
Error:
System.InvalidOperationException was unhandled
Message="The table specified in the SELECT statement does not contain a unique key or identifier column, or the SELECT statement does not include all of the key columns."
Since I have no way of controlling what the PK is on the destination DB, is there a way to tell the Adapter what the key is for this particular update? I have "custom" set the primary keys for each DataTable in the Dataset.
This is a non user interfacing batch process and its perf requirements are quite low. (to explain the use of datasets, etc)
Any Thoughts?

profile the db and see what queries are being fired. copy the queries from Profiler and try to exe them manually. if they fail, then the problem is with the sql and the error is coming from the db and simply being passed up to your application.
this should at least tell u if the problem is caused by the lack of real PK's in the db.

you have to set the PK for each table in the dataset like you did.
i don't think there's a way for adapter to figure out what the PK's are by itself.
if there is and someone knows it i'm all ears.

Maintain a local copy of a table from an external database table, ADO.NET

We have built an application which needs a local copy of a table from another database. I would like to write an ado.net routine which will keep the local table in sync with the master. Using .net 2.0, C# and ADO.NET.
Please note I really have no control over the master table which is in a third party, mission critical app I don't wish to mess with.
For example Here is the master data table:
ProjectCodeId Varchar(20) [PK]
ProjectCode Varchar(20)
ProjectDescrip Varchar(50)
OtherUneededField int
OtherUneededField2 int
The local table we need to keep in sync...
ProjectCodeId Varchar(20) [PK]
ProjectCode Varchar(20)
ProjectDescrip Varchar(50)
Perhaps a better approach to this question is what have you done in the past to this type of problem? What has worked best for you or should be avoided at all costs?
My goal with this question is to determine a good way to handle this. So often I am combining data from two or more disjointed data sources. I haven't included database platforms for this reason, it really shouldn't matter. In this current situation both databases are MSSQL, but I prefer the solution not use linked databases or DTS, etc.
Sure, truncating the local table and refilling it each time from the master is an option, but with thousands of rows I don't think this is very efficient. Do you?

EDIT: First, recognize that what you are doing is hand-rolled replication and replication is never simple.
You need to track and apply all of the CRUD state changes. That said, ADO.NET can do this.
To track changes to the source you can use Query Notification with your source database. This requires special permission against the database so the owner of the source database will need to take action to enable this solution. I haven't used this technique myself, but here is a description of it.
See "Query Notifications in SQL Server (ADO.NET)"
Query notifications were introduced in
Microsoft SQL Server 2005 and the
System.Data.SqlClient namespace in
ADO.NET 2.0. Built upon the Service
Broker infrastructure, query
notifications allow applications to be
notified when data has changed. This
feature is particularly useful for
applications that provide a cache of
information from a database, such as a
Web application, and need to be
notified when the source data is
changed.
To apply changes from the source db table you need to retrieve the data from the target db table, apply the changes to the target rows and post the changes back to the target db.
To apply the changes you can either
1) Delete and reinsert all of the rows (simple), or
2) Merge row-by-row changes (hard).
Delete and reinsert is self explanatory, so I won't go into detail on that.
For row-by-row change tracking here is an approach. (I am assuming here that Query Notification doesn't give you row-by-row change information, so you have to calculate it.)
You need to determine which rows were modified and identify inserted and deleted rows. Create a DataView with a sort for each table to get a Find method you can use to lookup matching rows by ID.
Identify modified rows by using a datetime/timestamp column, or by comparing all field values. Copy modified values to the target row.
Identify added and deleted rows by looping over the respective table DataViews and using the Find method of the other DataView to identify rows that do not appear in the first table. Insert or delete rows from the target table as required. (The Delete method doesn't remove the row but marks it for deletion by the TableAdapter Update.)
Good luck!
+tom

I would push in the direction where the application that is inserting the data would insert into one db/table then the other in the same function. Make the application do the work, the db will be pushed already.

Some questions - what db platform? how are you using the data?
I'm going to assume you're just using this data as a lookup... and as you have no timestamp and no ability modify the existing table, i'd just blow away the local copy periodically and pull it down from the master table again.
Unless you've got a hell of a lot of data the overhead for this should be pretty small.
If you need to synch back to the master table, you'll need to do something a bit more exotic.

Can you use SQL replication? This would be preferable to writing code to do it no?