Guys. I need an advice from you.
I created project MVC 5 and there user is importing excel file to (sql table) through interface on page. To give other users information about statuses.
In excel file 3 column ( in my model same columns )
Id NoAccount Status
User which uploading every week gets an excel file and importing file(s) using interface on page.
In excel file same columns may contain same data or newer data (with different statuses each week)
For example on 1st week user importing excel file with following data
Id NoAccount Status
1 A12345 0
On 2nd week user importing excel file with the next data.
Id NoAccount Status
1 A12345 1
For that moment, i will have, in my sql table 2 rows
And in a page also will be 2 rows and for preventing to confuse user should see only one row with status 1 (or 2,3 if it was )
If you want to keep multiple rows per account in your database and you want to see the highest value of status for each account, you can get the data using SQL like this
select NoAccount, max(Status)
from table
group by NoAccount
You mention T-SQL. You can use GROUP BY or SELECT DISTINCT constructs depending on the other fields that you want to include. So, for example:
SELECT DISTINCT
Id, NoAccount, Status
FROM
MyTable
or
SELECT
Id, NoAccount, Status, COUNT(Id)
FROM
MyTable
GROUP BY
Id, NoAccount, Status
would give you the distinct values together with the count of duplicates.
Related
I am working on a dynamic loader. Based on a database table that defines the flat text files I can read a single file with multiple record types and load it into database tables. The tables are related and using identity primary keys. Everything is currently working but runs really slow as would be expected given that it is all accomplished by single insert statements. I am working on optimizing the process and cant find an 'easy' or 'best practice' answer on the web.
My current project deals with 8 tables but to simplify I will use a customers / orders example.
Lets look at two customers below, the data would repeat for each set of customers and orders in the data file. Parent records are always before child records. The first field is record type and each record type has a different definition of the fields that follow. This is all specified in the control tables.
CUST|Joe Green|123 Main St
ORD|Pancakes|5
ORD|Nails|2
CUST|John Deere|456 Park Pl
ORD|Tires|4
Current code will:
Insert customer Joe Green and return an ID. (Using Output
Inserted.Id in the insert statement)
Insert orders pancakes and nails attaching the returned ID.
Insert customer John Deere and return an ID.
Insert order Tires with the return ID.
This runs painfully slow. If this could be optimized and I wouldn't have to change much code, that would be ideal but I cant think of how.
So the solution? I was thinking datatables... Here is what I am thinking of so far.
Create Transaction
Lock all tables that are part of the 'file definition', in this case
Customers and Orders Get max ID for each table and increment by one
to have starting IDs for all tables
Create datatable for all tables
Execute as currently set up but instead of issuing insert statements
add to data table
After data is read bulk upload tables in the correct order based on
relationships
Unlock tables
End Transaction
I was wondering, before I go down this path, if anyone has worked out a better solution. I am also considering a custom script component in SSIS. I have seen posts and blogs about holding off on commiting a transaction but each parent record has only a few child records and the tree can get up to 4 deep, think order details and products. Due to needing the parent record ID I need to commit the insert of parent records. I have also considered managing the ID's myself rather than Identity but I do not want to add that extra management if I can avoid it.
UPDATE based on answer, for clarification / context.
A typical text file has
one file header record
- 5 facility records that relate to the file header
- 7,000 customers(account)
- 5 - 10 notes per customer
- 1-5 payments at the account level
- 1-5 adjustments at the account level
- 5 - 20 orders per customer
- 5 - 20 order details per order
- 1-5 payments at the order level
- 1-5 adjustments at the order level
- one file trailer record related to the file header
Keys
- File Header -> Facility -> Customer (Account)
- File Header -> FileTrailer
- Customer -> Notes
- Customer -> Payments
- Customer -> Adjustments
- Customer -> Orders
- Order -> OrderDetails
- Order -> Payments
- Order -> Adjustments
There are a few more tables involved but this should give an idea of the overall context.
Data Sample ... = MORE FIELDS .... MORE RECORDS
HEADER|F1|F2|...
FACILITY|F1|F2|..
CUSTOMER|F1|F2|...
NOTE|F1|F2|....
....
ORDER|F1|F2|...
ORDERDETAIL|F1|F2|...
.... ORDER DETAILS
ORDERPYMT|F1|F2|...
....
ORDERADJ|F1|F2|...
....
CUSTOMERPYMT|F1|F2|...
....
CUSTOMERADJ|F1|F2|...
....
(The structure repeats for each facility)
TRAILER|F1|F2|...
Inserting related tables with low data volumes should normally not be a problem. If they are slow, we will need more context to answer your question.
If you are encountering problems because you have many records to insert, you will probably have to look at SqlBulkCopy.
If you prefer not managing your ids yourself, the cleanest way I know of is working with temporary placeholder id columns.
Create and fill datatables with your data and a tempId columns you fill yourself and foreign keys blank
SqlBulkCopy primary table
Update secondary datatable with generated foreign keys by finding primary keys from previously inserted table through your tempids column
Upload secondary table
Repeat until done
Remove temporary id columns (optional)
I have two tables in SQL Server DB. I have a dropdownmenu on my c# page where staff should be able to choose the relevant plan and it should show all users who do not have a current plan setup
Table 1 contains all users
Table 2 only contains people who have a plan. They also can have different plan_id's in table 2 from 1-10.
I need a SQL join which returns all people who do not have a plan at all or the plan id does not match the one selected from the dropdownlist
E.G a user could exist on table 2 with 3 entries with plan_id's 2,3 and 4.
E.G If a user selects plan 7 from the downdownlist, the user would appear in the list as they currently do not have a plan matching the id selected.
Table 2 plans also have an expiry_date and "active" field.
The plan should also not be valid if the expiry date is in the past or active is "0"
Write a not in Query, Where user does not exist in Table 2 with that plan id.
Suppose
Table_1 -> Userid, Username
Table_2 -> UserId, PlanId
SELECT UserId, Username FROM Table_1
WHERE UserId NOT IN ( SELECT UserId FROM Table_2 WHERE PlanId = 1 )
This query will return all the users who are not linked with Plan 1.
I am working on a database which has around 2 year data and has around 100 million rows and 30 columns with values of every 10 seconds of different parameters. I want to create a new table which will have average of these data containing only 1 row for each date of data. The database has around 100 000 rows for each date.
Table name is process
and primary key is id
How can I do it because whenever I search for something in this already existing table it takes a long time to find out the required output.
is it possible to create a new table which will take the average of all the data(around 1 lakh rows) of a single date and put them in one row
You want something like:
CREATE TABLE averages AS
SELECT
date_trunc('day', data_capture_timestamp_column) AS day,
avg(col1) AS col1_avg,
avg(col2) AS col2_avg,
...
FROM my_table
GROUP BY 1;
The GROUP BY 1 says to GROUP the data by the first SELECT argument, which in this case is the date. An expression index on my_table( date_trunc('day', data_capture_timestamp_column)) is recommended.
I have 3 tables in my Crystal Report XI for VS2010 and here's the scene:
The 3 tables linked together with a POID.
Table 1
Data: City, Date, Mill Name, POID (e.g. 1001)
Each POID has one record
Table 2
Data: Shade Name (e.g. Blue), Quantity (e.g. 123), POID (e.g. 1001)
Each POID has one or more records
Table 3
Data: Style Number (e.g. 123), Buyer's PO Number (e.g. 123), POID (e.g. 1001)
Each POID has one or more records
Table 2 and Table 3 are independent and only rely on Table 1 for their POID. The problem starts when there are differing number of records for a POID in Table 2 and Table 3.
Before Table 3 was added to the report, I grouped the report on the Table 1 POID and put the Table 2 record information in the Details section of the report. However, now that I have inserted the Table 3 records, the Table 2 and Table 3 records keep duplicating.
For example: for POID = 1001 there is one Table 1 record, three Table 2 records, and two Table 3 records. So, the outcome on the report is that each Table 2 record is repeated twice and each Table 3 record is repeated three times.
I don't want them to repeat themselves. I want them to display as they are on database.
I have used the POID group all of the table data using C#, VS2010, and SQLExpress2008.
I hope this is clear and that you can help. Thanks in advance.
Note: I did research a lot for this problem and then finally, asked here.
First of all, I think you're misinterpreting how a query on such a database would look. It's actually giving you exactly what a database query would return in a view if you have your joins set up as something like SELECT * FROM TABLE1 CROSS JOIN TABLE2 ON TABLE1.POID = TABLE2.POID CROSS JOIN TABLE3 ON TABLE2.POID = TABLE3.POID which means: if there is one TABLE1 record, three TABLE2 records, and two Table3 records, you'd be selecting the TABLE1 record six times, each TABLE2 record two times for a total of six, and each TABLE3 record three times for a total of six. So, really, you're returning 6 lines but everything in the report is FUBAR because of how you have it laid out. Grouping on TABLE1 records is probably the only "correct" thing you did.
I think the easiest way to handle without this problem without creating additional database objects is subreports. Hopefully this will make sense and I can throw together an image if you want, but what you want to do is this:
In your main report, using your main database object as the data source, group on TABLE1 and add two additional sections within the group header or footer (your choice) and suppress the details section entirely.
In the first subsection you added to the grouping, add a subreport using only TABLE2 as the data source, use no groupings and put your TABLE2 information in the detail section. Add additional header and summary information to the report header and footer as you deem necessary.
In the second subsection you added to the grouping, add a subreport using only TABLE3 as the data source, use no groupings and put your TABLE3 information in the detail section. Add additional header and summary information to the report header and footer as you deem necessary.
Add any additional information to the footers that you want.
In essence it should look something like:
Main Report - Report Header
Main Report - Page Header
Main Report - Group 1 Header
Group 1 Header - Section A - TABLE1 Record Information
Group 1 Header - Section B - SubReport 1
SubReport 1 - Report Header
SubReport 1 - Details - TABLE2 Record Information
SubReport 1 - Report Footer
Group 1 Header - Section C - SubReport 2
SubReport 2 - Report Header
SubReport 2 - Details - TABLE3 Record Information
SubReport 2 - Report Footer
Main Report - Details (Suppressed)
Main Report - Group 1 Footer
Main Report - Report Footer
Main Report - Page Footer
This will make a report with a TABLE1 header for each TABLE1 record followed by its TABLE2 and TABLE3 information without them repeating and making you crazy.
Does this make sense or help at all?
I'm writing an SQL query that displays a deliveries details, plus in a seperate column, all the delivery item ID's seperated by commas. Both Deliveries and DeliveryItems are in a seperate table, so I guess I'll be using a Join to retrieve all items that link to that delivery, but how do I get them into a column I'll be displaying, and seperate each item with a comma?
Thanks
EDIT: Here's an idea of my table structure
Deliveries -
ID (PK)
...(other non relevant fields)...
DeliveryItems -
ID (PK)
Delivery (links to ID of delivery)
So for every item of that delivery, I want to display the item ID comma seperated in a column for that delivery.
you can do it by using PIVOT in SQL server 2005.
PIVOT is used for Transpose row to column or column to row, so in your case you can get data in row format by using simple join after that you can use PIVOT for to transpose data and then you can concatenate columns in a single column.
check this link, it has a good example.
There is a GROUP_CONCAT function in MySQL that does just that (take a look at the documentation). Look for something similar in your DBMS.