Related
I have the following query, and because of a lot of SUM function calls, my query is running too slow. I have a lot of records in my database and I would like to get a report from the current year and last year (Last 30 days, Last 90 days and last 365 days) for each one:
SELECT
b.id as [ID]
,d.[Title] as [Title]
,e.Class as [Class]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Current - Last 30 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Current - Last 30 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Current - Last 90 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Current - Last 90 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Current - Last 365 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Current - Last 365 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-13,GETDATE()) and a.DateCol <= DATEADD(MONTH,-12,GETDATE()) THEN a.col1 ELSE 0 END) as [Last year - Last 30 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-13,GETDATE()) and a.DateCol <= DATEADD(MONTH,-12,GETDATE()) THEN a.col2 ELSE 0 END) as [Last year - Last 30 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-5,GETDATE()) and a.DateCol <= DATEADD(QUARTER,-4,GETDATE()) THEN a.col1 ELSE 0 END) as [Last year - Last 90 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-5,GETDATE()) and a.DateCol <= DATEADD(QUARTER,-4,GETDATE()) THEN a.col2 ELSE 0 END) as [Last year - Last 90 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-2,GETDATE()) and a.DateCol <= DATEADD(YEAR,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Last year - Last 365 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-2,GETDATE()) and a.DateCol <= DATEADD(YEAR,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Last year - Last 365 Days Col2]
FROM
tb1 a
INNER JOIN
tb2 b on a.id=b.fid and a.col3 = b.col4
INNER JOIN
tb3 c on b.fid = c.col5
INNER JOIN
tb4 d on c.id = d.col6
INNER JOIN
tb5 e on c.col7 = e.id
GROUP BY
b.id, d.Title, e.Class
Does anyone have any idea how can I improve my query in order to run faster?
EDIT: I was encouraged to move the DATEADD function call to the where statement and load first two years first then filter them in columns, but I am not sure the suggested answer is executed and works, it could be found here: https://stackoverflow.com/a/59944426/12536284
If you agree with the above solution, please show me how can I apply it in my current query?
Just FYI, I am using this SP in C#, Entity Framework (DB-First), something like this:
var result = MyDBEntities.CalculatorSP();
As it has been mentioned already, the execution plan will be really helpful in this case. Based on what you've shown it seems you have extracted 12 columns of 15 total columns from tb1 (a),
so you can try to run your query without any join and just against the tb1 to see whether your query is working as expected. Since I can see nothing wrong with your SUM function calls, my best guess is you have an issue with your joins, I would suggest to do the following. You can start by excluding the last join for instance,
INNER JOIN tb5 e on c.col7 = e.id and any related usage of it like e.Class as [Class] and e.Class in your group by statement. We are not going to exclude it completely, this is just a
test to make sure whether the problem is with that or not, if your query runs better and as expected you can try to use a temp table as a workaround instead of the last join, something
like this:
SELECT *
INTO #Temp
FROM
(
select * from tb5
) As tempTable;
SELECT
b.id as [ID]
,d.[Title] as [Title]
,e.Class as [Class]
-- SUM Functions
FROM
tb1 a
INNER JOIN
tb2 b on a.id=b.fid and a.col3 = b.col4
INNER JOIN
tb3 c on b.fid = c.col5
INNER JOIN
tb4 d on c.id = d.col6
INNER JOIN
#Temp e on c.col7 = e.id
GROUP BY
b.id, d.Title, e.Class
Actually, Temporary tables are tables that exist temporarily on the SQL Server. The temporary tables are useful for storing the immediate result sets that are accessed multiple times. You can read more about it here https://www.sqlservertutorial.net/sql-server-basics/sql-server-temporary-tables/
And here https://codingsight.com/introduction-to-temporary-tables-in-sql-server/
Also I would strongly recommend, if you are using the Stored Procedure, set the NOCOUNT to ON, it can also provide a significant performance boost, because network traffic is greatly reduced:
SET NOCOUNT ON
SELECT *
INTO #Temp
-- The rest of code
Based on this:
SET NOCOUNT ON is a set statement which prevents the message which shows the number of rows affected by T-SQL query statements. This is used within stored procedures and triggers to avoid showing the affected rows message. Using SET NOCOUNT ON within a stored procedure can improve the performance of the stored procedure by a significant margin.
The best approach is to insert into a table variable/hash table (if the row count is small use a table variable or use a hash table if the row count is pretty much big).
Then update the aggregation and then finally select from the table variable or hash table. Looking into the query plan is necessary.
DECLARE #MYTABLE TABLE (ID INT, [Title] VARCHAR(500), [Class] VARCHAR(500),
[Current - Last 30 Days Col1] INT, [Current - Last 30 Days Col2] INT,
[Current - Last 90 Days Col1] INT,[Current - Last 90 Days Col2] INT,
[Current - Last 365 Days Col1] INT, [Current - Last 365 Days Col2] INT,
[Last year - Last 30 Days Col1] INT, [Last year - Last 30 Days Col2] INT,
[Last year - Last 90 Days Col1] INT, [Last year - Last 90 Days Col2] INT,
[Last year - Last 365 Days Col1] INT, [Last year - Last 365 Days Col2] INT)
INSERT INTO #MYTABLE(ID, [Title],[Class],
[Current - Last 30 Days Col1], [Current - Last 30 Days Col2],
[Current - Last 90 Days Col1], [Current - Last 90 Days Col2],
[Current - Last 365 Days Col1], [Current - Last 365 Days Col2],
[Last year - Last 30 Days Col1], [Last year - Last 30 Days Col2],
[Last year - Last 90 Days Col1], [Last year - Last 90 Days Col2],
[Last year - Last 365 Days Col1], [Last year - Last 365 Days Col2]
)
SELECT b.id ,d.[Title] ,e.Class ,0,0,0,0,0,0,0,0,0,0,0,0
FROM tb1 a
INNER JOIN tb2 b on a.id=b.fid and a.col3 = b.col4
INNER JOIN tb3 c on b.fid = c.col5
INNER JOIN tb4 d on c.id = d.col6
INNER JOIN tb5 e on c.col7 = e.id
GROUP BY b.id, d.Title, e.Class
UPDATE T
SET [Current - Last 30 Days Col1]=K.[Current - Last 30 Days Col1] ,
[Current - Last 30 Days Col2] =K.[Current - Last 30 Days Col2],
[Current - Last 90 Days Col1] = K.[Current - Last 90 Days Col1],
[Current - Last 90 Days Col2] =K.[Current - Last 90 Days Col2] ,
[Current - Last 365 Days Col1] =K.[Current - Last 365 Days Col1],
[Current - Last 365 Days Col2] =K.[Current - Last 365 Days Col2],
[Last year - Last 30 Days Col1] =K.[Last year - Last 30 Days Col1],
[Last year - Last 30 Days Col2] =K.[Last year - Last 30 Days Col2],
[Last year - Last 90 Days Col1] =K.[Last year - Last 90 Days Col1],
[Last year - Last 90 Days Col2] =K.[Last year - Last 90 Days Col2],
[Last year - Last 365 Days Col1] =K.[Last year - Last 365 Days Col1],
[Last year - Last 365 Days Col2]=K.[Last year - Last 365 Days Col2]
FROM #MYTABLE T JOIN
(
SELECT
b.id as [ID]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col1 ELSE 0 END),0) as [Current - Last 30 Days Col1]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col2 ELSE 0 END),0) as [Current - Last 30 Days Col2]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-1,GETDATE()) THEN a.col1 ELSE 0 END),0) as [Current - Last 90 Days Col1]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-1,GETDATE()) THEN a.col2 ELSE 0 END),0) as [Current - Last 90 Days Col2]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-1,GETDATE()) THEN a.col1 ELSE 0 END),0) as [Current - Last 365 Days Col1]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-1,GETDATE()) THEN a.col2 ELSE 0 END),0) as [Current - Last 365 Days Col2]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-13,GETDATE()) and a.DateCol <= DATEADD(MONTH,-12,GETDATE()) THEN a.col1 ELSE 0 END),0) as [Last year - Last 30 Days Col1]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-13,GETDATE()) and a.DateCol <= DATEADD(MONTH,-12,GETDATE()) THEN a.col2 ELSE 0 END),0) as [Last year - Last 30 Days Col2]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-5,GETDATE()) and a.DateCol <= DATEADD(QUARTER,-4,GETDATE()) THEN a.col1 ELSE 0 END),0) as [Last year - Last 90 Days Col1]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-5,GETDATE()) and a.DateCol <= DATEADD(QUARTER,-4,GETDATE()) THEN a.col2 ELSE 0 END),0) as [Last year - Last 90 Days Col2]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-2,GETDATE()) and a.DateCol <= DATEADD(YEAR,-1,GETDATE()) THEN a.col1 ELSE 0 END),0) as [Last year - Last 365 Days Col1]
,ISNULL(Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-2,GETDATE()) and a.DateCol <= DATEADD(YEAR,-1,GETDATE()) THEN a.col2 ELSE 0 END),0) as [Last year - Last 365 Days Col2]
FROM tb1 a
INNER JOIN tb2 b on a.id=b.fid and a.col3 = b.col4
INNER JOIN tb3 c on b.fid = c.col5
INNER JOIN tb4 d on c.id = d.col6
INNER JOIN tb5 e on c.col7 = e.id
GROUP BY b.id
) AS K ON T.ID=K.ID
SELECT *
FROM #MYTABLE
I assume tb1 is a large table (relative to tb2, tb3, tb4 and tb5).
If so, it makes sense here to restrict the selection of that table (with a WHERE clause).
If only a small part of tb1 is used, for example because the joins with tb2, tb3, tb4 and tb5 reduce the needed rows to just a few percent, then you should check if the tables are indexed on the columns you use in the joins.
If a large part of tb1 is used, then it can make sense to group its results before joining it to tb2, tb3, tb4 and tb5. Below is an example of that.
SELECT
b.id as [ID]
,d.[Title] as [Title]
,e.Class as [Class]
,SUM(a.[Current - Last 30 Days Col1]) AS [Current - Last 30 Days Col1]
,SUM(a.[Current - Last 30 Days Col2]) AS [Current - Last 30 Days Col2]
,SUM(a.[Current - Last 90 Days Col1]) AS [Current - Last 90 Days Col1]
-- etc.
FROM (
SELECT a.id, a.col3
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Current - Last 30 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Current - Last 30 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Current - Last 90 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Current - Last 90 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Current - Last 365 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Current - Last 365 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-13,GETDATE()) and a.DateCol <= DATEADD(MONTH,-12,GETDATE()) THEN a.col1 ELSE 0 END) as [Last year - Last 30 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(MONTH,-13,GETDATE()) and a.DateCol <= DATEADD(MONTH,-12,GETDATE()) THEN a.col2 ELSE 0 END) as [Last year - Last 30 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-5,GETDATE()) and a.DateCol <= DATEADD(QUARTER,-4,GETDATE()) THEN a.col1 ELSE 0 END) as [Last year - Last 90 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(QUARTER,-5,GETDATE()) and a.DateCol <= DATEADD(QUARTER,-4,GETDATE()) THEN a.col2 ELSE 0 END) as [Last year - Last 90 Days Col2]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-2,GETDATE()) and a.DateCol <= DATEADD(YEAR,-1,GETDATE()) THEN a.col1 ELSE 0 END) as [Last year - Last 365 Days Col1]
,Sum(CASE WHEN a.DateCol >= DATEADD(YEAR,-2,GETDATE()) and a.DateCol <= DATEADD(YEAR,-1,GETDATE()) THEN a.col2 ELSE 0 END) as [Last year - Last 365 Days Col2]
FROM tb1 a
WHERE a.DateCol >= DATEADD(YEAR,-2,GETDATE())
GROUP BY a.id, a.col3
) AS a
INNER JOIN
tb2 b on a.id=b.fid and a.col3 = b.col4
INNER JOIN
tb3 c on b.fid = c.col5
INNER JOIN
tb4 d on c.id = d.col6
INNER JOIN
tb5 e on c.col7 = e.id
GROUP BY
b.id, d.Title, e.Class
Just use computed colums
Example
ALTER TABLE tb1 ADD [Current - Last 30 Days Col1] AS (CASE WHEN a.DateCol >= DATEADD(MONTH,-1,GETDATE()) THEN a.col1 ELSE 0 END) PERSISTED;
Specify Computed Columns in a Table
For optimizing such calculations you man consider pre-calculating some of the values. The idea of pre-calculations is to reduce the number of rows that need to be read or proceed.
One way of achieving this is using an indexed view and leave the engine to do the calculations by itself. As this type of views have some limitations, you man end up creating a simple table and perform the calculations instead. Basically, it depends on the business needs.
So, in the example below I am creating a table with RowID and RowDatetime columns and inserting 1 million rows. I am using an indexed view to count the entities per days, so instead of querying 1 million rows per year I will query 365 rows per year to count these metrics.
DROP TABLE IF EXISTS [dbo].[DataSource];
GO
CREATE TABLE [dbo].[DataSource]
(
[RowID] BIGINT IDENTITY(1,1) PRIMARY KEY
,[RowDateTime] DATETIME2
);
GO
DROP VIEW IF EXISTS [dbo].[vw_DataSource];
GO
CREATE VIEW [dbo].[vw_DataSource] WITH SCHEMABINDING
AS
SELECT YEAR([RowDateTime]) AS [Year]
,MONTH([RowDateTime]) AS [Month]
,DAY([RowDateTime]) AS [Day]
,COUNT_BIG(*) AS [Count]
FROM [dbo].[DataSource]
GROUP BY YEAR([RowDateTime])
,MONTH([RowDateTime])
,DAY([RowDateTime]);
GO
CREATE UNIQUE CLUSTERED INDEX [IX_vw_DataSource] ON [dbo].[vw_DataSource]
(
[Year] ASC,
[Month] ASC,
[Day] ASC
);
GO
DECLARE #min bigint, #max bigint
SELECT #Min=1 ,#Max=1000000
INSERT INTO [dbo].[DataSource] ([RowDateTime])
SELECT TOP (#Max-#Min+1) DATEFROMPARTS(2019, 1.0 + floor(12 * RAND(convert(varbinary, newid()))), 1.0 + floor(28 * RAND(convert(varbinary, newid()))) )
FROM master..spt_values t1
CROSS JOIN master..spt_values t2
GO
SELECT *
FROM [dbo].[vw_DataSource]
SELECT SUM(CASE WHEN DATEFROMPARTS([Year], [Month], [Day]) >= DATEADD(MONTH,-1,GETDATE()) THEN [Count] ELSE 0 END) as [Current - Last 30 Days Col1]
,SUM(CASE WHEN DATEFROMPARTS([Year], [Month], [Day]) >= DATEADD(QUARTER,-1,GETDATE()) THEN [Count] ELSE 0 END) as [Current - Last 90 Days Col1]
,SUM(CASE WHEN DATEFROMPARTS([Year], [Month], [Day]) >= DATEADD(YEAR,-1,GETDATE()) THEN [Count] ELSE 0 END) as [Current - Last 365 Days Col1]
FROM [dbo].[vw_DataSource];
The success of such solution depends very much on how the data is distributed and how many rows you have. For example, if you have one entry per day for each day of the year, the view and the table will have same match of rows, so the I/O operations will not be reduced.
Also, the above is just an example of materializing the data and reading it. In your case you may need to add more columns the view definition.
I would use a lookup table "Dates" table to join my data to with an index on DatesId. I use the dates as a filter when I want to browse historical data. The join is fast and so it the filtering as the DatesId is clustered primary index (primary key). Add the date column (as included column) for your data table as well.
The dates table has the following columns:
DatesId, Date, Year, Quarter, YearQuarter, MonthNum, MonthNameShort, YearWeek, WeekNum, DayOfYear, DayOfMonth, DayNumOfWeek, DayName
Example data:
20310409 2031-04-09 2031 2 2031-Q2 4 April Apr 2031_15 15 99 9 3 Wednesday
You can PM me if you want a csv of this so that you can import it to the database, but I'm sure you can easily find something like this online and make your own.
I add an identity column as well so that you can get an integer for each date. This makes it a bit easier to work with, but not a requirement.
SELECT * FROM dbo.dates where dateIndex BETWEEN (getDateIndexDate(getDate())-30 AND getDateIndexDate(getDate())+0) --30 days ago
This allows me to easily jump back to a certain period. It's quite easy to create your own views on this. You can of course use the ROW_NUMBER() function to do this for years, weeks, etc. as well.
Once I have the daterange I want, I join to the data. Works very fast!
Since you are always grouping values based on a whole number of months, I would first group by month in a subquery in the from clause. This is similar to using a temporary table. Not certain if this would actually speed up your query.
SELECT f.id, f.[Title], f.Class,
SUM(CASE WHEN f.MonthDiff = 1 THEN col1 ELSE 0 END) as [Current - Last 30 Days Col1],
-- etc
FROM (
SELECT
b.id,
d.[Title],
e.Class,
DateDiff(Month, a.DateCol, GETDATE()) as MonthDiff,
Sum(a.col1) as col1,
Sum(a.col2) as col2
FROM tb1 a
INNER JOIN tb2 b on a.id = b.fid and a.col3 = b.col4
INNER JOIN tb3 c on b.fid = c.col5
INNER JOIN tb4 d on c.id = d.col6
INNER JOIN tb5 e on c.col7 = e.id
WHERE a.DateCol between DATEADD(YEAR,-2,GETDATE() and GETDATE()
GROUP BY b.id, d.Title, e.Class, DateDiff(Month, a.DateCol, GETDATE())
) f
group by f.id, f.[Title], f.Class
To improve the speed of SQL query, you must add indexes. For each joined table, you have to add one index.
Like this code example for oracle:
CREATE INDEX supplier_idx
ON supplier (supplier_name);
I am working in HRMS. My problem is in vacation,
If an employee take vacation from 19/3/2014 to 5/4/2014 application calculates that he take 18 days in month instead of 3 days.
I store vacations in vacation table
columns :
emp_id | vac_type | from | to
Now, How can I make query to tell me that he take 13 days in March and 5 days in April?
Excellent question! I was able to find a way to do it, but I did have to use slightly different notation for the dates (see below).
DECLARE #startDate DATETIME, #endDate DATETIME, #lastDayOfStartMonth INT
SET #startDate = '3/19/2014'
SET #endDate = '4/5/2014'
SELECT #lastDayOfStartMonth =
1+DATEPART(dd, DATEADD(s,-1,DATEADD(mm, DATEDIFF(m,0,#startDate)+1,0)))
SELECT DATENAME(month, #startDate) AS [Month],
#lastDayOfStartMonth - DATEPART(dd, #startDate) AS [DaysSpent],
DATENAME(month, #endDate) AS [Month],
DATEPART(dd, #endDate) AS [DaysSpent]
Output:
| Month | DaysSpent | Month | DaysSpent |
|-------|-----------|-------|-----------|
| March | 13 | April | 5 |
SQL Fiddle example
My work here was based on the design from Pinal Dave's post SQL SERVER – Find Last Day of Any Month – Current Previous Next
Handling dates over more than two months
DECLARE #startDate DATETIME, #endDate DATETIME, #currentDate DATETIME, #currentDay INT
DECLARE #currentMonth INT, #lastDayOfStartMonth INT
CREATE TABLE #VacationDays ([Month] VARCHAR(10), [DaysSpent] INT)
SET #startDate = '1/19/2014'
SET #endDate = '4/5/2014'
SET #currentMonth = DATEPART(mm, #startDate)
SET #currentDay = DATEPART(dd, #startDate)
SET #currentDate = #startDate
WHILE #currentMonth < DATEPART(mm, #endDate)
BEGIN
SELECT #lastDayOfStartMonth =
DATEPART(dd, DATEADD(s,-1,DATEADD(mm, DATEDIFF(m,0,#currentDate)+1,0)))
PRINT #lastDayOfStartMonth
INSERT INTO #VacationDays
SELECT DATENAME(month, #currentDate) AS [Month],
#lastDayOfStartMonth - #currentDay + 1 AS [DaysSpent]
SET #currentDate = DATEADD(mm, 1, #currentDate)
SET #currentMonth = #currentMonth + 1
SET #currentDay = 1
END
IF DATEPART(mm, #startDate) = DATEPART(mm, #endDate)
BEGIN
INSERT INTO #VacationDays
SELECT DATENAME(month, #endDate) AS [Month],
DATEPART(dd, #endDate) - DATEPART(dd, #startDate) + 1 AS [DaysSpent]
END
ELSE
BEGIN
INSERT INTO #VacationDays
SELECT DATENAME(month, #endDate) AS [Month],
DATEPART(dd, #endDate) AS [DaysSpent]
END
SELECT * FROM #VacationDays
DROP TABLE #VacationDays
Output:
| Month | DaysSpent |
|----------|-----------|
| January | 13 |
| February | 28 |
| March | 31 |
| April | 5 |
SQL Fiddle example - It takes about a minute to run. It's much faster running in a local instance of SSMS.
Here's how it works
For the example below I am using a #startDate value of 05-05-2015.
The value of CAST(0 AS DATETIME) is the date 1900-01-01, so that means the line DATEDIFF(m,0,#startDate) is essentially asking, how many months have passed since January 1, 1900? For this example, that value is 1384.
DATEADD(mm, DATEDIFF(m,0,#startDate)+1,0) or DATEADD(mm, 1384+1,0) is saying, Add 1385 months to the date value 0 (or 1900-01-01). This will give us the DATETIME value of the first of the month after #startDate's month. For our example, 2015-06-01.
DATEADD(s,-1,DATEADD(mm, DATEDIFF(m,0,#startDate)+1,0)) or DATEADD(s,-1,'2015-06-01') subtracts 1 second from the first of next month, give us the last second of the current month, or 2015-05-31 23:59:59.
Then we use DATEPART to get the day value of that date: 31.
31 is the last day in May.
I don't know how to do this directly in SQL using a SQL query. But if you have the start and end dates of the vacation in your C# code, then you could calculate the number of days of vacation occurring in each month by doing something like the following. This got a little more complicated than I had intended, but it's the best I could come up with. For your example, this code produces the following output:
Vacation days in March: 13 days
Vacation days in April: 5 days
Code:
class Program
{
class DateRange
{
public DateTime Start { get; set; }
public DateTime End { get; set; }
}
static void Main(string[] args)
{
DateRange vacation = new DateRange();
vacation.Start = new DateTime(2014, 3, 19);
vacation.End = new DateTime(2014, 4, 5);
// Assuming April 5 represents the last day of vacation, let's
// add one to it, to show that his vacation actually ends on the
// following day.
vacation.End = vacation.End.AddDays(1);
DateRange currentMonth = new DateRange();
currentMonth.Start = new DateTime(vacation.Start.Year, vacation.Start.Month, 1);
currentMonth.End = currentMonth.Start.AddMonths(1);
while (currentMonth.Start < vacation.End)
{
Console.WriteLine("Vacation days in {0}: \t{1} days",
currentMonth.Start.ToString("MMMM"),
IntersectDates(currentMonth, vacation));
currentMonth.Start = currentMonth.Start.AddMonths(1);
currentMonth.End = currentMonth.End.AddMonths(1);
}
}
// Returns the number of days represented by the intersection of the two
// date ranges.
static int IntersectDates(DateRange dateRange1, DateRange dateRange2)
{
DateTime startOfIntersection = MaxDate(dateRange1.Start, dateRange2.Start);
DateTime endOfIntersection = MinDate(dateRange1.End, dateRange2.End);
return (startOfIntersection < endOfIntersection) ?
(int)(endOfIntersection - startOfIntersection).TotalDays :
0;
}
static DateTime MinDate(DateTime d1, DateTime d2)
{
return (d1 < d2) ? d1 : d2;
}
static DateTime MaxDate(DateTime d1, DateTime d2)
{
return (d1 > d2) ? d1 : d2;
}
}
DECLARE #FromDate date = '2014-03-19', #ToDate date = '2014-05-04'
WITH CTE AS
(
SELECT
YEAR(#FromDate) * 100 + MONTH(#FromDate) AS Month,
DATEADD(DAY, -DAY(#FromDate) + 1, #FromDate) AS FirstDateOfMonth,
DATEADD(DAY, -1, DATEADD(MONTH, 1, DATEADD(DAY, -DAY(#FromDate) + 1, #FromDate))) AS LastDateOfMonth
UNION ALL
SELECT
YEAR(DATEADD(MONTH, 1, FirstDateOfMonth)) * 100 + MONTH(DATEADD(MONTH, 1, FirstDateOfMonth)) AS Month,
DATEADD(MONTH, 1, FirstDateOfMonth),
DATEADD(DAY, -1, DATEADD(MONTH, 2, FirstDateOfMonth))
FROM CTE
WHERE #ToDate >= LastDateOfMonth
)
SELECT
*,
CASE
-- Same month
WHEN YEAR(#FromDate) * 100 + MONTH(#FromDate) = YEAR(#ToDate) * 100 + MONTH(#ToDate) THEN DATEDIFF(DAY, #FromDate, #ToDate) + 1
-- Get day from vacation start date to last date of month
WHEN Month = YEAR(#FromDate) * 100 + MONTH(#FromDate) THEN DATEDIFF(DAY, #FromDate, LastDateOfMonth) + 1
-- Get day from first date of month to vacation end date
WHEN Month = YEAR(#ToDate) * 100 + MONTH(#ToDate) THEN DATEDIFF(DAY, FirstDateOfMonth, #ToDate) + 1
-- Full month day
ELSE DATEDIFF(DAY, FirstDateOfMonth, LastDateOfMonth) + 1
END AS Day
FROM CTE
Result
Month FirstDateOfMonth LastDateOfMonth Day
----------- ---------------- --------------- -----------
201403 2014-03-01 2014-03-31 13
201404 2014-04-01 2014-04-30 30
201405 2014-05-01 2014-05-31 4
For applications like these, a dates table can greatly simplify queries at a reasonable cost to performance. It becomes nothing more than
SELECT emp_id, d.month, COUNT(*) as days
FROM Vacations v
INNER JOIN dates d on d.theDate BETWEEN v.Start and v.End
GROUP BY emp_id, d.month
See Create Date Dimension Table in SQL Server for an example of how to create a dates table.
This also simplifies complicated queries like the number of business days in a vacation:
SELECT emp_id, COUNT(*) as days
FROM Vacations v
INNER JOIN dates d on d.theDate BETWEEN v.Start and v.End
WHERE d.IsWeekend = 0
GROUP BY emp_id
Or grouping by pay-period or quarter:
SELECT emp_id, d.year, d.quarter, COUNT(*) as days
FROM Vacations v
INNER JOIN dates d on d.theDate BETWEEN v.Start and v.End
GROUP BY emp_id, d.year, d.quarter
I create a query that needs a calendar that I make it in WITH statement, In this calendar you can set your month names and month days -for a leap year-.
The calendar will be flexible for any year that used in vacation table -that automatically generate the correct dates for leap years-, Also support vacations starts from one year to next year.
;WITH calendar AS (
SELECT years.[Year], months.monthId, months.[monthName],
CASE ISDATE(CONVERT(varchar, years.[Year])+'-'+CONVERT(varchar,months.monthId)+'-'+CONVERT(varchar,months.monthDays))
WHEN 1 THEN months.monthDays
ELSE months.monthDays-1
END AS monthDays,
CONVERT(datetime, (CONVERT(varchar, years.[Year])+'-'+CONVERT(varchar,months.monthId)+'-1')) AS startDay,
CASE ISDATE(CONVERT(varchar, years.[Year])+'-'+CONVERT(varchar,months.monthId)+'-'+CONVERT(varchar,months.monthDays))
WHEN 1 THEN CONVERT(datetime, CONVERT(varchar, years.[Year])+'-'+CONVERT(varchar,months.monthId)+'-'+CONVERT(varchar,months.monthDays))
ELSE CONVERT(datetime, CONVERT(varchar, years.[Year])+'-'+CONVERT(varchar,months.monthId)+'-'+CONVERT(varchar,months.monthDays-1))
END AS EndDay
FROM
(SELECT DISTINCT YEAR(vi.[from]) As [Year] FROM vacation vi
UNION
SELECT DISTINCT YEAR(vi.[to]) As [Year] FROM vacation vi
) As years
CROSS JOIN
(SELECT 1 As monthId, 31 As monthDays, 'January' As [monthName] UNION ALL
SELECT 2, 29, 'February' UNION ALL
SELECT 3, 31, 'March' UNION ALL
SELECT 4, 30, 'April' UNION ALL
SELECT 5, 31, 'May' UNION ALL
SELECT 6, 30, 'June' UNION ALL
SELECT 7, 31, 'July' UNION ALL
SELECT 8, 31, 'August' UNION ALL
SELECT 9, 30, 'September' UNION ALL
SELECT 10, 31, 'October' UNION ALL
SELECT 11, 30, 'November' UNION ALL
SELECT 12, 31, 'December' ) As months
)
SELECT c.[year], c.monthId, c.[monthName],
CASE
WHEN v.emp_id IS NULL THEN 0
WHEN c.monthId = MONTH(v.[from]) THEN DATEDIFF(DAY, v.[from], c.EndDay) + 1
WHEN c.monthId = MONTH(v.[to]) THEN DATEDIFF(DAY, c.startDay, v.[to]) + 1
WHEN c.monthId BETWEEN MONTH(v.[from]) AND MONTH(v.[to]) THEN c.monthDays
END As vacationDays,
CASE
WHEN v.emp_id IS NULL THEN c.monthDays
WHEN c.monthId = MONTH(v.[from]) THEN DATEDIFF(DAY, c.startDay, v.[from])
WHEN c.monthId = MONTH(v.[to]) THEN DATEDIFF(DAY, v.[to], c.EndDay)
WHEN c.monthId BETWEEN MONTH(v.[from]) AND MONTH(v.[to]) THEN 0
END As nonvacationDays,
c.monthDays
FROM
calendar c
LEFT JOIN
vacation v ON c.monthId BETWEEN MONTH(v.[from]) AND MONTH(v.[to])
The result will be like this:
year | monthId | monthName | vacationDays | nonvacationDays | monthDays
-----+---------+-----------+--------------+-----------------+------------
2014 | 1 | January | 0 | 31 | 31
2014 | 2 | February | 0 | 28 | 28
2014 | 3 | March | 13 | 18 | 31
2014 | 4 | April | 5 | 25 | 30
2014 | 5 | May | 0 | 31 | 31
--....
I'm looking for a fast and easy SQL Query to get records year by year, month by month and day by day.
My database example:
ID - DATE - CatID - VALUE
1 - 2013-08-06 - 32 - 243
2 - 2013-08-16 - 2 - 45
3 - 2013-08-21 - 2 - 1
4 - 2013-08-05 - 32 - 450
5 - 2013-08-05 - 32 - 449
6 - 2013-08-05 - 32 - 11
7 - 2013-08-01 - 2 - 221
8 - 2013-08-02 - 32 - 0
9 - 2013-08-02 - 32 - 0
10 - 2013-08-02 - 32 - 987
...
..
I have over 2 millions records on this table.
First:
I would like to get CatID = 32 and single/higher record of the day.
Result:
1 - 2013-08-06 - 32 - 243
4 - 2013-08-05 - 32 - 450
10 - 2013-08-02 - 32 - 987
Second:
I would like to get as same but by MONTH.
How can I do it with SQL SERVER. Currently Sql Server 2012.
Also using C# 5 and if you want to use, EF 6.
Down voters, this question became "popular question". Now please explain why down voted?
The below assumes that you want more than just the max value for each date/month, but also want to know the id of that row, etc, etc.
For daily...
WITH
sorted AS
(
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY CatID,
Date
ORDER BY Value DESC) AS ordinal_cat_date
FROM
yourTable
)
SELECT
*
FROM
sorted
WHERE
ordinal_cat_date = 1
AND catID = 32
For monthly...
WITH
sorted AS
(
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY CatID,
DATEADD(MONTH, DATEDIFF(MONTH, 0, Date), 0)
ORDER BY Value DESC) AS ordinal_cat_month
FROM
yourTable
)
SELECT
*
FROM
sorted
WHERE
ordinal_cat_month = 1
AND catID = 32
In both cases the ROW_NUMBER() function creates a sequence of numbers (1,2,3,4,etc) for each PARTITION of catID, date, in descending order of value. Whichever row has a value of 1 is the row with the highest value in that partition.
The formula DATEADD(MONTH, DATEDIFF(MONTH, 0, Date), 0) just rounds the Date down to the first day of the month, thereby creating partitions for whole months.
Some other options that bring through multiple rows per day/month if those rows all share the same highest value...
SELECT -- Daily Version
*
FROM
yourTable
WHERE
NOT EXISTS (SELECT *
FROM yourTable AS lookup
WHERE lookup.CatID = yourTable.CatID
AND lookup.Value > yourTable.Value
AND lookup.Date = yourTable.Date
)
AND CatID = 32
SELECT -- Monthly Version
*
FROM
yourTable
WHERE
NOT EXISTS (SELECT *
FROM yourTable AS lookup
WHERE lookup.CatID = yourTable.CatID
AND lookup.Value > yourTable.Value
AND lookup.Date >= DATEADD(MONTH, DATEDIFF(MONTH, 0, yourTable.Date), 0)
AND lookup.Date < DATEADD(MONTH, 1+DATEDIFF(MONTH, 0, yourTable.Date), 0)
)
AND CatID = 32
Or possibly...
SELECT -- Daily Version
*
FROM
yourTable
INNER JOIN
(
SELECT
Date,
CatID,
MAX(Value) AS max_value
FROM
yourTable
GROUP BY
CatID,
Date
)
AS lookup
ON yourTable.Date = lookup.Date
AND yourTable.CatID = lookup.CatID
AND yourTable.Value = lookup.max_value
WHERE
yourTable.CatID = 32
SELECT -- Monthly Version
*
FROM
yourTable
INNER JOIN
(
SELECT
CatID,
DATEADD(MONTH, DATEDIFF(MONTH, 0, yourTable.Date), 0) AS month_start,
MAX(Value) AS max_value
FROM
yourTable
GROUP BY
CatID,
DATEADD(MONTH, DATEDIFF(MONTH, 0, yourTable.Date), 0)
)
AS lookup
ON yourTable.Date >= lookup.month_start
AND yourTable.Date < DATEADD(MONTH, 1, lookup.month_start)
AND yourTable.CatID = lookup.CatID
AND yourTable.Value = lookup.max_value
WHERE
yourTable.CatID = 32
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
we are using Punch Machine for Attendance with zkemkeeper.dll and geting AttLog using Vs.net 2010.How can Get Records from SQL Query?
my table structure like this
LogID int P.K.
DeviceIP varchar
EnrollNo int
AttDate datetime
AttYear int
AttMonth int
AttDay int
AttTime varchar
My current output:
LogID EnrollNo AttDate AttYear AttMonth AttDay AttTime
1 319 1/9/2011 9:55:00 PM 2011 8 31 9:55
2 319 1/9/2011 18:30:00 PM 2011 8 31 18:30
3 325 1/9/2011 10:00:00 PM 2011 8 31 10:00
4 325 1/9/2011 18:35:00 PM 2011 8 31 18:35
I want the new output like this with calculation of in-out time difference:
LogID EnrollNo AttDate AttYear AttMonth AttDay In out Diff
1 319 1/9/2011 9:55:00 PM 2011 8 31 9:55 18:30 8:35
2 325 1/9/2011 6:30:00 PM 2011 8 31 10:00 18:35 8:35
declare #t table (EnrollNo int, [Date] datetime, Time varchar(5))
insert #t select 1, '8-10-2011 12:00:32', '13:12'
union all select 1, '8-10-2011 12:00:32', '23:14'
union all select 2, '8-10-2011 12:00:32', '11:12'
union all select 2, '8-10-2011 12:00:32', '20:14'
union all select 3, '8-10-2011 12:00:35', '12:12'
union all select 3, '8-10-2011 12:00:32', '23:14'
union all select 4, '8-10-2011 12:00:32', '17:12'
union all select 4, '8-10-2011 12:00:32', '23:14'
select
EnrollNo,
CAST(CONVERT(varchar, Date, 101) AS DateTime),
right('0' + cast(datediff(hour, cast(min(Time) as datetime),
cast(max(Time) as datetime)) as varchar(2)),2) + ':' +
right('0' + cast(datediff(minute, cast(min(Time) as datetime),
cast(max(Time) as datetime)) % 60 as varchar(2)),2),
min(Time),
max(Time)
from #t group by EnrollNo,CAST(CONVERT(varchar, Date, 101) AS DateTime)
Try this...
SELECT TIMEDIFF("18:30","9:55") as Diff;
or
SELECT TIMEDIFF(In,Out) as Diff;
Output: 08:35:00
Do you want the first and last times for the same EnrolNo on the same day? Try a Group By to get you 'In' and 'Out' fields, then use the timediff from other answers to calculate the difference:
SELECT EnrolNo, AttYear, AttMonth, AttDay, Min(AttDate) AS [In], Max(AttDate) AS [Out],
TIME_Format(TIMEDIFF([Out],[In]),'%H:%i') As [Diff]
FROM Table
GROUP BY EnrolNo, AttYear, AttMonth, AttDay;
I have a textfield that users input the date using a modified jquery datepicker. The textfield has the Month and Year such as 'July 2011'
I need to run a query to search for results between July 1, 2011 and July 31, 2011. My date in the database is a smalldatetime, so I need my query to look like this:
select * from members where signupdate between 'july 1 2011' and 'july 31 2011'
How can I get the user inputted date of July 2011 converted to 'July 1 2011' and 'July 31 2011'?
EDIT
I'm only getting a 0 value from InvalidCount but I know I have one record in there as a test. Why isn't it being counted?
MY PROC:
SELECT
(SELECT COUNT(*) FROM dbo.Members m WHERE m.memberID = #pMemberID AND m.SignUpDate BETWEEN #pDate AND DATEADD(MONTH, 1, #pDate)-1) AS 'SignUpDate',
COALESCE(SUM(m.ValidCount), 0) AS ValidCount,
COALESCE(SUM(m.InvalidCount), 0) AS InvalidCount
FROM
dbo.Members m
INNER JOIN
dbo.MemberStats ms ON m.MemberID = ms.MemberID
WHERE
m.SignUpdate BETWEEN #pDate AND DATEADD(MONTH, 1, #pDate)-1
The exact syntax depends on the SQL Engine, but if you start with the 1st of the month, then add 1 month, and finally subtract 1 day; you get the end of the month.
(I'll assume MS SQL Server to match your C# tag)
SELECT
*
FROM
members
WHERE
signupdate BETWEEN #param AND DATEADD(MONTH, 1, #param) - 1
If the between isn't required, you can use DatePart.
Untested example:
where DATEPART(yyyy, SignupDate) = 2011 and DATEPART(m, SignupDate) = 7
convert varchar to appropiate format you want and then compare
check the list of formats. Hope this helps dude.
convert date
Firstly, you must convert the string representation to a valid DateTime struct object on the server. You can use var date = DateTime.ParseExact("july 1 2011", "MMMM dd yyyy", CultureInfo.CurrentUICulture) or DateTime.TryParse. Then you pass this into your SqlCommand as parameters. Never use strings when querying, especially when it comes from user input.