Out of my lack of SQL Server experience and taking into account that this task is a usual one for Line of Business applications, I'd like to ask, maybe there is a standard, common way of doing the following database operation:
Assume we have two tables, connected with each other by one-to-many relationship, for example SalesOderHeader and SalesOrderLines
http://s43.radikal.ru/i100/1002/1d/c664780e92d5.jpg
Field SalesHeaderNo is a PK in SalesOderHeader table and a FK in SalesOrderLines table.
In a front-end app a User selects some number of records in the SalesOderHeader table, using for example Date range, or IsSelected field by clicking checkbox fields in a GridView. Then User performs some operations (let it be just "move to another table") on selected range of Sales Orders.
My question is:
How, in this case, I can reach child records in the SalesOrderLines table for performing the same operations (in our case "move to another table") over these child records in as easy, correct, fast and elegant way as possible?
If you're okay with a T-SQL based solution (as opposed to C# / LINQ) - you could do something like this:
-- define a table to hold the primary keys of the selected master rows
DECLARE #MasterIDs TABLE (HeaderNo INT)
-- fill that table somehow, e.g. by passing in values from a C# apps or something
INSERT INTO dbo.NewTable(LineCodeNo, Item, Quantity, Price)
SELECT SalesLineCodeNo, Item, Quantity, Price
FROM dbo.SalesOrderLine sol
INNER JOIN #MasterIDs m ON m.HeaderNo = sol.SalesHeaderNo
With this, you can insert a whole set of rows from your child table into a new table based on a selection criteria.
Your question is still a bit vague to me in that I'm not exactly sure what would be entailed by "move to another table." Does that mean there is another table with the exact schema of both your sample tables?
However, here's stab at a solution. When a user commits on a SalesOrderHeader record, some operation will be performed that looks like:
Update SalesOrderHeader
Set....
Where SalesOrderHeaderNo = #SalesOrderHeaderNo
Or
Insert SomeOtherTable
Select ...
From SalesOrderHeader
Where SalesOrderHeaderNo = #SalesOrderHeaderNo
In that same operation, is there a reason you can't also do something to the line items such as:
Insert SomeOtherTableItems
Select ...
From SalesOrderLineItems
Where SalesOrderHeaderNo = #SalesOrderHeaderNo
I don't know about "Best Practices", but this is what I use:
var header = db.SalesOrderHeaders.SingleOrDefault(h => h.SaleHeaderNo == 14);
IEnumerable<SalesOrderLine> list = header.SalesOrderLines.AsEnumerable();
// now your list contains the "many" records for the header
foreach (SalesOrderLine line in list)
{
// some code
}
I tried to model it after your table design, but the names may be a little different.
Now whether this is the "best practices" way, I am not sure.
EDITED: Noticed that you want to update them all, possibly move to another table. Since LINQ-To-SQL can't do bulk inserts/updates, you would probably want to use T-SQL for that.
Related
I have a DataGrid View pulling some items from my database. What I want to achieve is to be able to edit the pack size or the bar_code fields. I am aware on how to update values in a database but how would I go about doing it if the data is the same? Meaning in many instances a bar code would have multiple pack sizes that is related to the one bar code number. Let's say I have the below screenshot. A data entry error was made and the bar_code and PackSize columns are the exact same. I want to change the first bar code to "1234." How would I achieve this? I can't say update barcode to 'textBox1.Text' where bar_code = '771313166386' because it would then change both data. How do I go about only focusing on one row of data at a time?
You can try using this query to update only the first row:
UPDATE TOP (1) my_table
SET bar_code = '1234'
WHERE bar_code = '771313166386'
You should have an auto-increment id column or a Primary key in your table.
I'd suggest you handle the logic of data duplicate manipulation at the backend rather than pulling them inside the grid and handle it there.
The following query will help you retrieve the duplicate records based on the mentioned columns. You can change it to UPDATE or DELETE as per your requirement.
-- Using cte and ranking function
;With CTE
As
(
Select
Product,
Description,
BarCode,
PackSize
Row_Number() Over(Partition By Product, BarCode, PackSize Order By Product) As RowNum
From YourTable
)
Select * From CTE
-- Where RowNum > 1;
Hope this is helpful :)
This might not help you directly in your answer. But, it is important to mention that your table design is incorrect. You should ensure the data integrity by creating a primary key in your table.
So when you need to update a product you have only one row to update.
Then you can add more tables and use foreign key references between them.
You need to uniquely represent the products. As per your sample data, I guess that there isn't any primary key on your table.
What you can do is either specify a unique constraint on columns to ensure that this type of data entry cannot be done.
If you cannot come up with list of columns to uniquely identify the rows, you can use surrogate keys by specifying Identity column and then while updating, always put a constraint where thisIdentityColumn=value
A data entry error was made and the bar_code and PackSize columns are
the exact same
I think this is the key. Essentially, the exact duplicates are unintentional, and the rows should be unique. Further it looks like bar_code + pack_size is your primary key (subject to data being entered correctly).
So, when you do an update, simply update the first row found that matches a bar_code and a pack_size. If it isn't unique, then the update should ensure that you are one step closer to unique rows in the database.
If you need a non-verbal answer, let me know.
i have a sql server database with table. These are
1stAP_TB, 2ndAP_TB, 3rdAP_TB, 4thAP_TB, 1steng_TB, 2ndeng_TB, 3rdeng_TB,
4theng_TB
all in them are in row. The numbers will be solve individually on specific column. Now, i need to know how am i going to get the average of 1stAP_TB, 2ndAP_TB, 3rdAP_TB and 4thAP_TB while there are in rows.
Also, there are multiple data that will be save inside the database. I am using C# programming language.
Try below method
create table aveexample
(a1stAP_TB int,
a2ndAP_TB int,
a3rdAP_TB int,
a4thAP_TB int,
a1steng_TB int,
a2ndeng_TB int,
a3rdeng_TB int,
a4theng_TB int
)
Sample data
insert into aveexample values(1,2,3,4,5,6,7,8)
insert into aveexample values(11,22,33,44,55,66,77,78)
insert into aveexample values(2,3,1,4,10,10,45,5)
Method 1
select *, (select AVG(totaldata)
from (values(a1stAP_TB),
(a2ndAP_TB),(a3rdAP_TB),(a4thAP_TB),(a1steng_TB),
(a2ndeng_TB),(a3rdeng_TB),(a4theng_TB)) total(totaldata))as average
from aveexample
Method 2
select ((a1stAP_TB)+
(a2ndAP_TB)+(a3rdAP_TB)+(a4thAP_TB)+(a1steng_TB)+
(a2ndeng_TB)+(a3rdeng_TB)+(a4theng_TB))/8 as Average
from aveexample
It is difficult to give concrete advice given the very limited description in the question, but from the description and comments so far, it seems to me like the database needs to be redesigned to better fit your requirements. First, you have no ID field, so there is no way to differentiate one row from the next. Then, what you are left with is a series of repeated values. The clue here is that you have "1st", "2nd", "3rd" in the column names. That's probably a sign that those columns need to be moved into rows of a related table. It may not instantly seem to be the best approach, but this is called "First Normal Form" and is a typical best practice with SQL databases. See also Database Normalization Basics.
It seems to me that what you have here is some entity (which you haven't mentioned in your question) that has a number of values associated with it. The 'entity' here should be given a unique ID and then all of the values for that entity stored with its ID.
You might have a table with the following columns:
CREATE TABLE MyItems (
ID int NOT NULL,
Sequence int NOT NULL,
Value int NOT NULL,
CONSTRAINT PK_MyValues_ID_Sequence PRIMARY KEY
(ID,Sequence)
)
Note: ID + sequence forms the unique primary key for the table and makes every row unique. This also lets you keep track of what order the items were added in. This may or may not be important to you but every table should probably have a unique primary key.
Your data table would then look something like this (the example represents two different entities, the first having 4 values and the second having 3 values):
It's difficult to show a sensible example without knowing more about the application and what it does... but with this table design you have a basis from which to add values one at a time, as you said you needed, and a way to query them back. You can use grouping to produce things like totals and averages, or you can do that in code by iterating over the results of a query or in a LINQ statement.
You can then compute the average for an entity of a given ID using a LINQ query along the lines of:
var average = MyItems.Where(p=>p.ID == 1).Average(q=>q.Value);
As an example of the flexibility of this sort of approach, you could just as easily compute the average of every second value entered across the entire database:
var averageOfSecondItems = MyItems.Where(p => p.Sequence == 2).Average(q => q.Value);
The example I've shown deals with one type of value. In your question it appears that you might have two different types of value. There are several ways you could handle that - for example you could add another column to the table if the values are always entered in pairs, or you could create a second table to hold the separate values. Again, it's hard to make a recommendation based on the limited information given.
If putting your data into First Normal Form seems like a lot of work, then your application might be a better fit for a document database ("NoSQL" database), but that is really a different question. In the question, a SQL database was specified so I've concentrated on that.
I use asp.net 4 and DataSets for accessing the database. There are two tables with one-to-one relationship in the database. It means that both tables have the same column as a primary key (say Id), and one of tables has #identity on this column set.
So in general if we want to insert, we insert first into the first table, than insert into the second table with id.table2 = id of the corresponding record in table1.
I can imagine how to achieve this using stored procedure (we would insert into the first table and have id as an out parameter and then insert into the second table using this id, btw all inside one transaction).
But is there a way to do it without using a stored procedure? May be DataSets \ DataAdapters have such functionality built in?
Would appreciate any help.
Today it is so quiet here... Ok if anybody is also looking for such a solution, I've found a way to do it.
So our main problem is to get the id of the newly created record in the first table. If we're able to do that, after that we simply supply it to the next method which creates a corresponding record in the second table.
I used a DataSet Designer in order to enjoy the code autogeneration feature of the VS. Let's call the first table TripSets. In DataSet Designer right click on the TripSetsTableAdapter, then Properties. Expand InsertCommand properties group. Here we need to do two things.
First we add a new parameter into the collection of parameters using the Parameters Collection Editor. Set ParameterName = #TripId, DbType = Int32 (or whatever you need), Direction = Output.
Second we modify the CommandText (using Query Builder for convenience). Add to the end of the command another one after a semicolon like that:
(...);
SELECT #TripId = SCOPE_IDENTITY()
So you will get something like this statement:
INSERT INTO TripSets
(Date, UserId)
VALUES
(#Date,#UserId);
SELECT #TripId = SCOPE_IDENTITY()
Perhaps you will get a parser error warning, but you can just ignore it. Having this configured now we are able to use in our Business logic code as follows:
int tripId;
int result = tripSetsTableAdapter.Insert(tripDate, userId, out tripId);
// Here comes the insert method into the second table
tripSetTripSearchTableAdapter.Insert(tripId, amountPersons);
Probably you will want to synchronize this operations somehow (e.g. using TransactionScope) but it is completely up to you.
I need to update a bit field in a table and set this field to true for a specific list of Ids in that table.
The Ids are passed in from an external process.
I guess in pure SQL the most efficient way would be to create a temp table and populate it with the Ids, then join the main table with this and set the bit field accordingly.
I could create a SPROC to take the Ids but there could be 200 - 300,000 rows involved that need this flag set so its probably not the most efficient way. Using the IN statement has limitation wrt the amount of data that can be passed and performance.
How can I achieve the above using the Entity Framework
I guess its possible to create a SPROC to create a temp table but this would not exist from the models perspective.
Is there a way to dynamically add entities at run time. [Or is this approach just going to cause headaches].
I'm making the assumption above though that populating a temp table with 300,000 rows and doing a join would be quicker than calling a SPROC 300,000 times :)
[The Ids are Guids]
Is there another approach that I should consider.
For data volumes like 300k rows, I would forget EF. I would do this by having a table such as:
BatchId RowId
Where RowId is the PK of the row we want to update, and BatchId just refers to this "run" of 300k rows (to allow multiple at once etc).
I would generate a new BatchId (this could be anything unique -Guid leaps to mind), and use SqlBulkCopy to insert te records onto this table, i.e.
100034 17
100034 22
...
100034 134556
I would then use a simgle sproc to do the join and update (and delete the batch from the table).
SqlBulkCopy is the fastest way of getting this volume of data to the server; you won't drown in round-trips. EF is object-oriented : nice for lots of scenarios - but not this one.
I'm assigning Marcs response as the answer but I'd just like to give a little detail on how we implemented the requirement.
Marc response helped greatly in the formulation of our solution.
We had to deal with an aim/guideline to keep within the Entity Framework while not utilizing SPROCS and although our solution may not suit others it has worked for us
We created a Item table in the Database with BatchId [uniqueidentifier] and ItemId varchar columns.
This table was added to the EF model so we did not use temporary tables.
On upload of these Ids this table is populated with the Ids [Inserts are quick enough we find using EF]
We then use context.ExecuteStoreCommand to run the SQL to do join the item table and the main table and update the bit field in the main table for records that exist for the batch Id created specifically for that session.
We finally clear this table for that batchId.
We have the performance, keeping within our no SPROC goal. [Which not of us agree with :) but its a democracy]
Our exact requirements are a little more complex but insofar as needing good update performance using the Entity framework given our specific restrictions it works fine.
Liam
Using the ADO.NET MySQL Connector, what is a good way to fetch lots of records (1000+) by primary key?
I have a table with just a few small columns, and a VARCHAR(128) primary key. Currently it has about 100k entries, but this will become more in the future.
In the beginning, I thought I would use the SQL IN statement:
SELECT * FROM `table` WHERE `id` IN ('key1', 'key2', [...], 'key1000')
But with this the query could be come very long, and also I would have to manually escape quote characters in the keys etc.
Now I use a MySQL MEMORY table (tempid INT, id VARCHAR(128)) to first upload all the keys with prepared INSERT statements. Then I make a join to select all the existing keys, after which I clean up the mess in the memory table.
Is there a better way to do this?
Note: Ok maybe its not the best idea to have a string as primary key, but the question would be the same if the VARCHAR column would be a normal index.
Temporary table: So far it seems the solution is to put the data into a temporary table, and then JOIN, which is basically what I currently do (see above).
I've dealt with a similar situation in a Payroll system where the user needed to generate reports based on a selection of employees (eg. employees X,Y,Z... or employees that work in certain offices). I've built a filter window with all the employees and all the attributes that could be considered as a filter criteria, and had that window save selected employee id's in a filter table from the database. I did this because:
Generating SELECT queries with dynamically generated IN filter is just ugly and highly unpractical.
I could join that table in all my queries that needed to use the filter window.
Might not be the best solution out there but served, and still serves me very well.
If your primary keys follow some pattern, you can select where key like 'abc%'.
If you want to get out 1000 at a time, in some kind of sequence, you may want to have another int column in your data table with a clustered index. This would do the same job as your current memory table - allow you to select by int range.
What is the nature of the primary key? It is anything meaningful?
If you're concerned about performance I definitely wouldn't recommend an 'IN' clause. It's much better try do an INNER JOIN if you can.
You can either first insert all the values into a temporary table and join to that or do a sub-select. Best is to actually profile the changes and figure out what works best for you.
Why can't you consider using a Table valued parameter to push the keys in the form of a DataTable and fetch the matching records back?
Or
Simply you write a private method that can concatenate all the key codes from a provided collection and return a single string and pass that string to the query.
I think it may solve your problem.