SQL Query taking too much time to execute. Working fine at UAT. I need to compare data of two tables and want to get difference. Below mention is my query.
Select *
from tblBrandDetailUsers tbdu
inner join tblBrands tbs on tbs.BrandId = tbdu.BrandId
left join tblBrandDetails tbd on tbd.CategoryId = tbdu.CategoryId
and tbd.BrandId = tbdu.BrandId
and tbd.CityId = tbdu.CityId
inner join tblCategory tc on tbdu.CategoryId = tc.CategoryId
inner join tblCity tcc on tcc.CityId = tbdu.CityId
where isnull(tbdu.SaleAmount,-1) <> isnull(tbd.SaleAmount,-1)
and isnull(tbdu.CityId,0) = 3
and isnull(tbdu.TopLevelCategoryId,0) = 2;
Need to optimize query.
a number of things you need to check:
number of rows for each table. the more rows you have the slower it gets. Do you have the same size of data with UAT?
SELECT * : avoid the * and only retrieve columns you need.
ISNULL function on left side of the WHERE predicate will scan the index because it is non-sargable. you can check the answer here and rewrite your predicate without any function on the left side of WHERE clause.
You need to provide a detailed information like actual execution plan. I can only give you a generic answer because not much detail was provided.
Remember the UAT is very different in PROD. the hardware you used, the number of rows, etc..
Every advises in comment looks right. The difference between UAT and Prod should be the volume of data.
Your issue should come of lack or inefficient indices.
You should add compound index on
tblBrandDetails.CategoryId,tblBrandDetails.BrandId, tblBrandDetails.CityId
and on
tblBrandDetailUsers.CategoryId,tblBrandDetailUsers.BrandId, tblBrandDetailUsers.CityId
ensure that all unique ids have a btree index (or similar type of index depending on your DB)
You can also add conditional indices to filter quicker the null values:
https://www.brentozar.com/archive/2015/09/filtered-indexes-and-is-not-null/
Rewrite your query like this :
Select *--> AVOID "*" put all the necessary columns
from tblBrandDetailUsers AS tbdu
inner join tblBrands AS tbs on tbs.BrandId = tbdu.BrandId
left join tblBrandDetails AS tbd on tbd.CategoryId = tbdu.CategoryId
and tbd.BrandId = tbdu.BrandId
and tbd.CityId = tbdu.CityId
inner join tblCategory AS tc on tbdu.CategoryId = tc.CategoryId
inner join tblCity AS tcc on tcc.CityId = tbdu.CityId
where tbdu.SaleAmount <> tbd.SaleAmount
and tbdu.CityId = 3
and tbdu.TopLevelCategoryId = 2
UNION ALL
SELECT * --> AVOID "*" put all the necessary columns
from tblBrandDetailUsers AS tbdu
inner join tblBrands AS tbs on tbs.BrandId = tbdu.BrandId
left join tblBrandDetails AS tbd on tbd.CategoryId = tbdu.CategoryId
and tbd.BrandId = tbdu.BrandId
and tbd.CityId = tbdu.CityId
inner join tblCategory AS tc on tbdu.CategoryId = tc.CategoryId
inner join tblCity AS tcc on tcc.CityId = tbdu.CityId
where tbdu.SaleAmount IS NULL
AND tbd.SaleAmount IS NULL
and tbdu.CityId = 3
and tbdu.TopLevelCategoryId = 2;
Modify the SELECT clause to have only the necessary columns and not *
Be sure that you have index that are close to :
For tblBrandDetailUsers TABLE :
index KEY (CityId, TopLevelCategoryId, BrandId, CategoryId) INCLUDE (SaleAmount)
index KEY (CityId, TopLevelCategoryId, CategoryId) INCLUDE (SaleAmount)
For tblBrandDetails TABLE :
index (CityId, BrandId, CategoryId)
And also :
tblCategory (CategoryId)
tblCity (CityId)
tblBrands (BrandId)
When you will rectify the query especially the SELECT clause, we can give you more accurate indexes, because selected columns have a big weight on indexes performances !
as other suggested, try to add index on columns used for joins
Related
I have an linq query like this :
var query = from Romm in RoMM
join rfrsa in RoMmfrsa on Romm.RoMmid equals rfrsa.RoMmid
join frsa in Frsa on rfrsa.Frsaid equals frsa.Fraid
join fra in Fra on frsa.Fraid equals fra.Fraid
where Romm.ActTypeId == 2 && Romm.SegmentId == 4
select new
{
Romm.ActTypeId,
Romm.RoMmid,
frsa.Fraid,
frsa.Frsaid,
Romm.ImpactId
};
And I have SQL code as below :
SELECT romm.ROMMID
, frsa.FRAID
, frsa.FRSAID
, romm.ImpactID
FROM RoMM AS romm
INNER
JOIN RoMMFRSA AS rfrsa
ON romm.RoMMID = rfrsa.RoMMID
INNER
JOIN FRSA AS frsa
ON rfrsa.frsaid = frsa.frsaid
INNER
JOIN FRA AS fra
ON frsa.FRAID = fra.FRAID
WHERE romm.acttypeid = 2
AND romm.segmentid = 4
The SQL only shows one row (which is correct), the linq shows the correct row and then it displays about another 3 rows which is not what we need. I need the linq to show one row which is correct with the SQL. Is this because of maybe many-many relationships ?
Looks like a typo in either the C# or the SQL join:
SQL: ON rfrsa.frsaid = frsa.frsaid
C#: rfrsa.Frsaid equals frsa.Fraid
^^^^^^
mismatch here
I have data in two tables and I need in one query get all data and join getting data.
SELECT
kpip.PersonalName,
kpiT.Name,
kpiPR.KpiTarget,
kpiPR.KpiResultDate,
kpiPR.KpiResult
FROM KpiPersonalResult AS kpiPR join KpiPersonal as kpip
on kpiPR.KpiPersonal = kpip.Id join KpiType AS kpiT
on kpip.KpiType = kpiT.Id join MerchantAdministrators as merA
on kpiPR.KpiAdded = merA.Id and kpiPR.KpiResultDate between '2021-04-07' and '2021-04-08'
select
kpiP.PersonalName,
kpiT.Name,
kpiP.KpiTarget
from KpiPersonal as kpiP join KpiType as kpiT
on kpiP.KpiType = kpiT.Id
Based on the fast that the second query has 3 columns of the same name as the first query, I guess you mean to union them:
SELECT
kpip.PersonalName,
kpiT.Name,
kpiPR.KpiTarget,
kpiPR.KpiResultDate,
kpiPR.KpiResult
FROM
KpiPersonalResult AS kpiPR
join KpiPersonal as kpip on kpiPR.KpiPersonal = kpip.Id
join KpiType AS kpiT on kpip.KpiType = kpiT.Id
join MerchantAdministrators as merA on kpiPR.KpiAdded = merA.Id and kpiPR.KpiResultDate between '2021-04-07' and '2021-04-08'
UNION ALL
select
kpiP.PersonalName,
kpiT.Name,
kpiP.KpiTarget,
null, --put suitable default values for the other columns here
null
from
KpiPersonal as kpiP
join KpiType as kpiT on kpiP.KpiType = kpiT.Id
Unioned queries need the same number of columns. I've inserted NULL as default value for the two missing columns in the second query (relative to the first)
UNION makes a resultset grow taller. If you intended for it to grow wider, that is done via JOIN. A simple pattern for doing so is:
WITH query1 AS(
--query 1 here
), query2 AS (
--query2 here
)
SELECT * FROM query1 JOIN query2 ON ...
Side note on formatting and indenting - most people find SQL most readable when all operations that are related are at the same indent level e.g in a typical query, the SELECT FROM WHERE GROUP ORDER keywords are all at the same indent level, with the blocks that relate to them (the list of selected columns, or list of joined tables, list of where'd predicates etc) indented a level again. We also typically don't use as when aliasing tables but we do use it when aliasing columns in the SELECT
I have put down below a query to retrieve from four tables which are
Sales_Invoice, New_Customer, Company_Information, Expense
Query:
select
isnull(sum(Expense.Expense_Amount), 0.00), as ExpenseAmount,
Company_Information.Company_Name,
Sales_Invoice.Invoice_No, Sales_Invoice.Invoice_Date, Sales_Invoice.Item_Name,
New_Customer.Customer_Name, New_Customer.Customer_ID
from
Sales_invoice, Company_Information, New_Customer, Expense
where
Sales_Invoice.Customer_Id = New_Customer.Customer_ID
and Sales_Invoice.Invoice_No = Expense.Invoice_No
group by
Company_Information.Company_Name,
Sales_Invoice.Invoice_No, Sales_Invoice.Invoice_Date, Sales_Invoice.Customer_ID,
Sales_Invoice.Item_Name, New_Customer.Customer_Name, New_Customer.Customer_ID
The query is working well but if the Expense table has no values Expense.Invoice_No does not match with Sales_Invoice.Invoice_No, then the query above will return empty rows.
But what I wish to do is that, if Expense.Invoice_No does not exists then I still want to have my rows provided that expense amount return 0.00
Use standard joins! Then, you can easily handle "missing" relations with a left join.
Your question suggests:
select
coalesce(sum(e.expense_amount), 0.00) as expenseamount,
ci.company_name,
si.invoice_no, si.invoice_date, si.item_name,
nc.customer_name, nc.customer_id
from new_customer nc
inner join company_information ci on ???
inner join sales_invoice si on si.customer_id = nc.customer_id
left join expense e on e.invoice_no = si.invoice_no
group by
ci.company_name,
si.invoice_no, si.invoice_date, si.customer_id,
si.item_name, nc.customer_name, nc.customer_id
Note that you original code seems to me missing a join condition between the customers and companies. I represented it as ??? in the query.
You could also express the same logic with a correlated subquery, which would avoid outer aggregation:
select
(
select coalesce(sum(e.expense_amount), 0.00)
from expense e
where e.invoice_no = si.invoice_no
) as expenseamount,
ci.company_name,
si.invoice_no, si.invoice_date, si.item_name,
nc.customer_name, nc.customer_id
from new_customer nc
inner join company_information ci on ???
inner join sales_invoice si on si.customer_id = nc.customer_id
Just to set the context a little, I'm trying to use queries with mysql that use Late row lookup as shown in this article
https://explainextended.com/2009/10/23/mysql-order-by-limit-performance-late-row-lookups/
but that's a story for another day but the idea is that you do a key search on the table and then join it onto the whole table to force a late row lookup and the problem is coming from my LINQ queries when joined together.
-- Key search query --
Calling Code
IQueryable<int> keySearch = _defaultQueryFactory.Load(ContextEnums.ClientContext, MapEntityToDTO(), whereStatement, clientID).OrderBy(orderBy).Skip(startRow).Take(pageSize).Select(x => x.ID);
Resulting Query
SELECT
`Extent1`.`Sys_InvoiceID`
FROM `tblinvoice` AS `Extent1`
WHERE 3 = `Extent1`.`FK_StatusID`
ORDER BY
`Extent1`.`InvoiceDate` ASC LIMIT 0,430
-- Full Table Search --
Calling Code
IQueryable<InvoiceDTOModel> tableSearch = _defaultQueryFactory.Load(ContextEnums.ClientContext, MapEntityToDTO(), null, clientID, true).OrderBy(orderBy);
Resulting Query
SELECT
`Extent1`.`ID`,
`Extent1`.`C1`,
`Extent1`.`C2`,
`Extent1`.`C3`,
`Extent1`.`C4`,
`Extent1`.`C5`,
`Extent1`.`C6`,
`Extent2`.`SID`,
`Extent2`.`S1,
`Extent2`.`S2`,
`Extent2`.`S3`,
`Extent3`.`EID`,
`Extent3`.`E1`,
`Extent4`.`DID`,
`Extent4`.`D1`,
`Extent4`.`D2`,
`Extent4`.`D3`,
`Extent4`.`D4`,
`Extent4`.`D5`
FROM `tbl1` AS `Extent1` INNER JOIN `tbl2` AS `Extent2` ON `Extent1`.`SID` = `Extent2`.`SID` INNER JOIN `tbl3` AS `Extent3` ON `Extent1`.`EID` = `Extent3`.`EID` LEFT OUTER JOIN `tbl4` AS `Extent4` ON `Extent1`.`ID` = `Extent4`.`DID`
ORDER BY
`Extent1`.`C4` ASC
-- Joining the Two Together --
Calling Code
keySearch.Join(tableSearch, key => key, table => table.ID, (key, table) => table).OrderBy(orderBy).ToListAsync();
Resulting Query
SELECT
`Join3`.`ID`,
`Join3`.`C1`,
`Join3`.`C1`,
`Join3`.`C1`,
`Join3`.`C1`,
`Join3`.`C1`,
`Join3`.`C1`,
`Join3`.`SID`,
`Join3`.`S1,
`Join3`.`S2`,
`Join3`.`S3`,
`Join3`.`EID`,
`Join3`.`E1`,
`Join3`.`DID`,
`Join3`.`D1`,
`Join3`.`D2`,
`Join3`.`D3`,
`Join3`.`D4`,
`Join3`.`D5`
FROM (
`Extent1`.`ID`,
`Extent1`.`C1`,
`Extent1`.`C2`,
`Extent1`.`C3`,
`Extent1`.`C4`,
`Extent1`.`C5`,
`Extent1`.`C6`
FROM `tblinvoice` AS `Extent1`
WHERE 3 = `Extent1`.`EID`
ORDER BY
`Extent1`.`C4` ASC LIMIT 0,430) AS `Limit1` INNER JOIN (SELECT
`Extent1`.`ID`,
`Extent1`.`C1`,
`Extent1`.`C2`,
`Extent1`.`C3`,
`Extent1`.`C4`,
`Extent1`.`C5`,
`Extent1`.`C6`,
`Extent2`.`SID`,
`Extent2`.`S1,
`Extent2`.`S2`,
`Extent2`.`S3`,
`Extent3`.`EID`,
`Extent3`.`E1`,
`Extent4`.`DID`,
`Extent4`.`D1`,
`Extent4`.`D2`,
`Extent4`.`D3`,
`Extent4`.`D4`,
`Extent4`.`D5`
FROM `tbl1` AS `Extent2` INNER JOIN `tbl2` AS `Extent3` ON `Extent2`.`SID` = `Extent3`.`SID` INNER JOIN `tblstatus` AS `Extent4` ON `Extent2`.`EID` = `Extent4`.`EID` LEFT OUTER JOIN `tbl3` AS `Extent5` ON `Extent2`.`ID` = `Extent5`.`DID`) AS `Join3` ON `Limit1`.`ID` = `Join3`.`ID`
ORDER BY
`Join3`.`C4` ASC
Basically the inner select brings back
FROM (
`Extent1`.`ID`,
`Extent1`.`C1`,
`Extent1`.`C2`,
`Extent1`.`C3`,
`Extent1`.`C4`,
`Extent1`.`C5`,
`Extent1`.`C6`
FROM `tblinvoice` AS `Extent1`
WHERE 3 = `Extent1`.`EID`
ORDER BY
`Extent1`.`C4` ASC LIMIT 0,430) AS `Limit1`
Instead of
FROM (
`Extent1`.`ID`,
FROM `tblinvoice` AS `Extent1`
WHERE 3 = `Extent1`.`EID`
ORDER BY
`Extent1`.`C4` ASC LIMIT 0,430) AS `Limit1`
--Note--
The actual query selects around 15 columns, I've just shortened it to this example, it has an effect on the search as the dataset grows in size and it shouldn't be selecting all of the fields but i suspect there's an error in my join.
Any help is much appreciated.
I"m having some problems with Nhibernate and native sql.
I've got an entity with alot of collections and I am doing an SQL Fulltext search on it. So when returning 100 or so entities, I dont want all collections be lazy loaded. For this I changed my SQL query:
SELECT Query.*
FROM (SELECT {spr.*},
{adr.*},
{adrt.*},
{cty.*},
{com.*},
{comt.*},
spft.[Rank] AS [Rak],
Row_number() OVER(ORDER BY spft.[Rank] DESC) AS rownum
FROM customer spr
INNER JOIN CONTAINSTABLE ( customerfulltext , computedfulltextindex , '" + parsedSearchTerm + #"' ) AS spft
ON spr.customerid = spft.[Key]
LEFT JOIN [Address] adr
ON adr.customerid = spr.customerid
INNER JOIN [AddressType] adrt
ON adrt.addresstypeid = adr.addresstypeid
INNER JOIN [City] cty
ON cty.cityid = adr.cityid
LEFT JOIN [Communication] com
ON com.customerid = spr.customerid
INNER JOIN [CommunicationType] comt
ON comt.communicationtypeid = com.communicationtypeid) as Query
ORDER BY Query.[Rank] DESC
This is how I setup the query:
var items = GetCurrentSession()
.CreateSQLQuery(query)
.AddEntity("spr", typeof(Customer))
.AddJoin("adr", "spr.addresses")
.AddJoin("adrt", "adr.Type")
.AddJoin("cty", "adr.City")
.AddJoin("com", "spr.communicationItems")
.AddJoin("comt", "com.Type")
.List<Customer>();
What happens now is, that the query returns customers twice (or more), I assume this is because of the joins since for each customer address, communicationItem (e.g. phone, email), a new sql row is returned. In this case I thought I could use the DistinctRootEntityResultTransformer.
var items = GetCurrentSession()
.CreateSQLQuery(query)
.AddEntity("spr", typeof(Customer))
.AddJoin("adr", "spr.addresses")
.AddJoin("adrt", "adr.Type")
.AddJoin("cty", "adr.City")
.AddJoin("com", "spr.communicationItems")
.AddJoin("comt", "com.Type")
.SetResultTransformer(new DistinctRootEntityResultTransformer())
.List<Customer>();
Doing so an exception is thrown. This is because I try to list customers .List<Customer>() but the transformer returns only entities of the last join added. E.g. in the case above, the entity with alias "comt" is returned when doing .List() instead of .List<Customer>(). If I would switch last join with the join alias "cty", then the transformer returns a list of cities only...
Anyone knows how I can return a clean list of customers in this case?
try this
var items = GetCurrentSession()
.CreateSQLQuery(query)
.AddEntity("spr", typeof(Customer))
.AddJoin("adr", "spr.addresses")
.AddJoin("adrt", "adr.Type")
.AddJoin("cty", "adr.City")
.AddJoin("com", "spr.communicationItems")
.AddJoin("comt", "com.Type")
.AddEntity("spr", typeof(Customer))
.SetResultTransformer(new DistinctRootEntityResultTransformer())
.List<Customer>();
GetCurrentSession.CreateQuery("select distinct spr from Customer spr inner join spr.Addresses
adr inner join adr.Type adrt inner join adr.City cty inner join apr.CommunicationItems com
inner join com.Type comt").List<Customer>();
Hope this helps. Or you can write the desired SQL query under
GetCurrentSession.CreateSqlQuery(sqlquery).List<Customer>();