Develop Reference

Basic misunderstanding of LINQ to SQL and foreign keys

I am working on a much larger project, and can't seem to get LINQ to SQL working the way I expect it to. I created a simple subset of the project so I can use LinqPad to try to make sure I have a basic understanding of how this should work.
Clearly, I don't: I've created two very simple tables - customer and job. The customer table has an ID (auto-increment) and a Name, the Job table has an ID (ai), a Name, and a CustomerID (foreign key to ID in the customer table).
When I run the following code against an initially empty database:
void Main()
{
string custName = "James";
string[] jobNames = new string[] {"Home Depot", "Menards", "Sam's Club" };
var cust = customer.FirstOrDefault(c => c.Name == custName);
if (cust == null)
{
cust = new customer
{
Name = custName
};
customer.InsertOnSubmit(cust);
}
foreach(var jn in jobNames)
{
if (!job.Any(j => j.Customer.Name == cust.Name && j.Name == jn))
job.InsertOnSubmit(new job {
Name = jn,
Customer = cust
});
}
SubmitChanges();
customer.Dump();
job.Dump();
}
I would expect to end up with 1 customer and 3 jobs in the database - that's all good. But the generated SQL and the setting of the customer IDs are not at all what I expect:
SQL --
SELECT t0.ID, t0.Name
FROM customer AS t0
WHERE (t0.Name = #p0)
LIMIT 0, 1
-- p0 = [James]
SELECT COUNT(*) AS value
FROM job AS t0
LEFT OUTER JOIN customer AS t1
ON (t1.ID = t0.CustomerID)
WHERE ((t1.Name = #p0) AND (t0.Name = #p1))
-- p0 = [James]
-- p1 = [Home Depot]
SELECT COUNT(*) AS value
FROM job AS t0
LEFT OUTER JOIN customer AS t1
ON (t1.ID = t0.CustomerID)
WHERE ((t1.Name = #p0) AND (t0.Name = #p1))
-- p0 = [James]
-- p1 = [Menards]
SELECT COUNT(*) AS value
FROM job AS t0
LEFT OUTER JOIN customer AS t1
ON (t1.ID = t0.CustomerID)
WHERE ((t1.Name = #p0) AND (t0.Name = #p1))
-- p0 = [James]
-- p1 = [Sam's Club]
INSERT INTO job(CustomerID, ID, Name)
VALUES (NULL, 0, #p0)
-- p0 = [Home Depot]
INSERT INTO job(CustomerID, ID, Name)
VALUES (NULL, 0, #p0)
-- p0 = [Menards]
INSERT INTO job(CustomerID, ID, Name)
VALUES (NULL, 0, #p0)
-- p0 = [Sam's Club]
INSERT INTO customer(ID, Name)
VALUES (0, #p0)
-- p0 = [James]
SELECT t0.ID, t0.Name
FROM customer AS t0
SELECT t0.CustomerID, t0.ID, t0.Name
FROM job AS t0
Results in LinqPad:
I thought that the beauty of LINQ to SQL was that I don't have to manage to set the foreign keys and that I should be able to do what I'm trying to do here with a single hit to the database. What am I missing?
EDIT: So I guess I understand the customer IDs being set to NULL because the generated SQL is calling INSERT on the jobs before the INSERT on the customer, hence there is no ID yet. Why would it do that? Also, if I run the same query again, I get three more rows in the jobs table, but the CustomerIDs are all still set to NULL.

Linq query have no issues.
check DB table relations

2 parameterized where clauses in a single query (not 'where _ and _')

Can parameterizing 2 different where clauses (non contiguous) in a single query be done in dapper?
Sample SQL query as C# property
private string GetSalesRepToCustomer => #"
WITH SALES_REP_FILTERED AS (
SELECT
SALES_REP_ID
FROM
SALES_REP
/**where**/ /* 1st where clause. */
)
SELECT
A.SALES_REP_ID,
B.CUSTOMER_ID
FROM
SALES_REP_FILTERED A LEFT JOIN
CUSTOMER B ON A.SALES_REP_ID = B.SALES_REP_ID
/**where**/ /* 2nd where clause */
";
C# parameterization not working, illustration purposes.
SqlTemplate queryTemplate = new SqlBuilder()
.where("SALES_REP_ID = :SALES_REP_ID")
// this 2nd 'where' is illustrative.
// In actual an 'and' clause is created.
// My goal is to have a secondary/non contiguous 'where' clause in the same query.
.where("CUSTOMER_ID IN :CUSTOMER_ID")
.AddTemplate(GetSalesRepToCustomer);
var conn = new OracleConnection();
var queryResults = conn.Query(
queryTemplate.RawSql,
new { SALES_REP_ID = 2021, CUSTOMER_ID = new int[] {11, 22, 33}}
);
Context and reasoning of what I want to achieve:
Read online (Cant recall where. Probably on stackoverflow) the incentive(s) for dapper query parameterization and would like to stick/follow to this recommended practice.
I could split the query up into 2 and make separate round trips to the DB. But there are performance and IO payoffs/benefits when executed as a single query.
My last option is to in line the where clauses in SQL. But this probably would negate the benefits of dapper parameterization? I.e.
private string GetSalesRepToCustomer => #"
WITH SALES_REP_FILTERED AS (
SELECT
SALES_REP_ID
FROM
SALES_REP
where SALES_REP_ID = :SALES_REP_ID
)
SELECT
A.SALES_REP_ID,
B.CUSTOMER_ID
FROM
SALES_REP_FILTERED A LEFT JOIN
CUSTOMER B ON A.SALES_REP_ID = B.SALES_REP_ID
WHERE CUSTOMER_ID IN :CUSTOMER_ID
";
And the where parameterization is taken out.
SqlTemplate queryTemplate = new SqlBuilder()
// No parameterized where clauses. So also no dapper performance improvements?
.AddTemplate(GetSalesRepToCustomer);
var conn = new OracleConnection();
var queryResults = conn.Query(
queryTemplate.RawSql,
new { SALES_REP_ID = 2021, CUSTOMER_ID = new int[] {11, 22, 33}}
);

As I understand it, this should be fine:
var conn = new OracleConnection();
var sql = #"
WITH SALES_REP_FILTERED AS (
SELECT
SALES_REP_ID
FROM
SALES_REP
where SALES_REP_ID = :SALES_REP_ID
)
SELECT
A.SALES_REP_ID,
B.CUSTOMER_ID
FROM
SALES_REP_FILTERED A LEFT JOIN
CUSTOMER B ON A.SALES_REP_ID = B.SALES_REP_ID
WHERE CUSTOMER_ID IN :CUSTOMER_ID
";
var queryResults = conn.Query<SOME_TYPE>(
sql,
new { SALES_REP_ID = 2021, CUSTOMER_ID = new int[] {11, 22, 33}}
);
But your SQL could also be written as:
SELECT
A.SALES_REP_ID,
B.CUSTOMER_ID
FROM
SALES_REP SR
LEFT JOIN CUSTOMER C ON SR.SALES_REP_ID = C.SALES_REP_ID
WHERE
C.CUSTOMER_ID IN :CUSTOMER_ID AND
SR.SALES_REP_ID = :SALES_REP_ID
If you pass 3 customer IDs, then dapper will rewrite your WHERE clause to be:
WHERE C.CUSTOMER_ID IN (:CUSTOMER_ID1, :CUSTOMER_ID2, :CUSTOMER_ID3) AND
and it will populate the command's parameters collection with the three parameters populated with the values supplied

Improving SQL query generated by LINQ to EF

I have student table and I want to write a Linq query to get multiple counts. The SQL query that Linq generates is too complicated and un-optimized.
Following is definition of my table:
[Id] [int] IDENTITY(1,1) NOT NULL,
[Name] [varchar](100) NULL,
[Age] [int] NULL,
I need to get one count with students with name = test and one count for students with age > 10.
This is one of the query I have tried:
var sql = from st in school.Students
group st by 1 into grp
select new
{
NameCount = grp.Count(k => k.Name == "Test"),
AgeCount = grp.Count(k => k.Age > 5)
};
The SQL query that is generated is:
SELECT
[Limit1].[C1] AS [C1],
[Limit1].[C2] AS [C2],
[Limit1].[C3] AS [C3]
FROM ( SELECT TOP (1)
[Project2].[C1] AS [C1],
[Project2].[C2] AS [C2],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[Student] AS [Extent3]
WHERE ([Project2].[C1] = 1) AND ([Extent3].[Age] > 5)) AS [C3]
FROM ( SELECT
[Distinct1].[C1] AS [C1],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[Student] AS [Extent2]
WHERE ([Distinct1].[C1] = 1) AND (N'Test' = [Extent2].[Name])) AS [C2]
FROM ( SELECT DISTINCT
1 AS [C1]
FROM [dbo].[Student] AS [Extent1]
) AS [Distinct1]
) AS [Project2]
) AS [Limit1]
For me this seems to be complex. This can be achieved by following simple query:
select COUNT(CASE WHEN st.Name = 'Test' THEN 1 ELSE 0 END) NameCount,
COUNT(CASE WHEN st.Age > 5 THEN 1 ELSE 0 END) AgeCount from Student st
Is there a way in LINQ with which the SQL query that gets generated will have both the aggregation rather than having it two separate queries joined with nested queries?

From my experience with EF6, conditional Sum (i.e. Sum(condition ? 1 : 0)) is translated much better to SQL than Count with predicate (i.e. Count(condition)):
var query =
from st in school.Students
group st by 1 into grp
select new
{
NameCount = grp.Sum(k => k.Name == "Test" ? 1 : 0),
AgeCount = grp.Sum(k => k.Age > 5 ? 1 : 0)
};
Btw, your SQL example should be using SUM as well. In order to utilize the SQL COUNT which excludes NULLs, it should be ELSE NULL or no ELSE:
select COUNT(CASE WHEN st.Name = 'Test' THEN 1 END) NameCount,
COUNT(CASE WHEN st.Age > 5 THEN 1 END) AgeCount
from Student st
But there is no equivalent LINQ construct for this, hence no way to let EF6 generate such translation. But IMO the SUM is good enough equivalent.

A much simpler query results from not using the unnecessary group by and just querying the table twice in the select:
var sql = from st in school.Students.Take(1)
select new {
NameCount = school.Students.Count(k => k.Name == "Test"),
AgeCount = school.Students.Count(k => k.Age > 5)
};

LINQ Query generates unnecessarily long SQL

I have a bunch of Services, who references Students (many to one), who references StudentEnrollments (one to many).
When I query these services, it is generating SQL that contains 2 blocks that looks the same which slows down performance. I cannot for the life of me figure out why.
Here is my C# Code (narrowed down):
IQueryable<StudentServiceDm> query = GetListQuery();
List<int> schoolIds = // from front-end: in this case: 20, 21, 22, 23, 89, 90, 93, 95
query = query.Where(m => m.Student.StudentEnrollments.Any(s => schoolIds.Contains(s.SchoolId.Value)));
IQueryable<StudentServiceDto> dtoQuery = query.Select(m => new StudentServiceDto
{
Id = m.Id,
Name = m.Name,
ParentParticipationCount = m.ParentCount,
StudentFirstName = m.Student.FirstName,
StudentLastName = m.Student.LastName,
StudentId = m.StudentId.Value,
StudentServiceType = m.StudentServiceType.Name,
StudentServiceSubType = m.StudentServiceSubType.Name,
Date = m.Date,
DurationInMinutes = m.DurationInMinutes
});
return dtoQuery;
Here is the generated SQL:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
[Extent1].[ParentCount] AS [ParentCount],
[Extent2].[FirstName] AS [FirstName],
[Extent2].[LastName] AS [LastName],
[Extent1].[StudentId] AS [StudentId],
[Extent3].[Name] AS [Name1],
[Extent4].[Name] AS [Name2],
[Extent1].[Date] AS [Date],
[Extent1].[DurationInMinutes] AS [DurationInMinutes]
FROM [dbo].[StudentService] AS [Extent1]
LEFT OUTER JOIN [dbo].[Student] AS [Extent2] ON ([Extent2].[Deleted] = 0) AND ([Extent1].[StudentId] = [Extent2].[Id])
LEFT OUTER JOIN [dbo].[StudentServiceType] AS [Extent3] ON ([Extent3].[Deleted] = 0) AND ([Extent1].[StudentServiceTypeId] = [Extent3].[Id])
LEFT OUTER JOIN [dbo].[StudentServiceSubType] AS [Extent4] ON ([Extent4].[Deleted] = 0) AND ([Extent1].[StudentServiceSubTypeId] = [Extent4].[Id])
WHERE ([Extent1].[Deleted] = 0) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[StudentEnrollment] AS [Extent5]
INNER JOIN [dbo].[Student] AS [Extent6] ON [Extent6].[Id] = [Extent5].[StudentId]
WHERE ([Extent6].[Deleted] = 0) AND ([Extent1].[StudentId] = [Extent6].[Id]) AND ([Extent5].[Deleted] = 0) AND ([Extent5].[SchoolId] IN (20, 21, 22, 23, 89, 90, 93, 95))
)) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[StudentEnrollment] AS [Extent7]
INNER JOIN [dbo].[Student] AS [Extent8] ON [Extent8].[Id] = [Extent7].[StudentId]
WHERE ([Extent8].[Deleted] = 0) AND ([Extent1].[StudentId] = [Extent8].[Id]) AND ([Extent7].[Deleted] = 0) AND ([Extent7].[SchoolId] IN (20, 21, 22, 23, 89, 90, 93, 95))
))
ORDER BY [Extent1].[Date] DESC, [Extent1].[Id] ASC
OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY
As you can see, the SQL is doing two boolean blocks (A AND B) where A and B looks exactly the same (with the [extend] suffix being different of course). I think my query is simple enough as to not confuse LINQ to generate such query. Can any expert tell me why this is happening? Or How I can write my query in another way.

Entity Framework makes little attempt to optimize the SQL being generated - quite the opposite in practice. It's meant to be convenient rather than fast.
LINQ and Entity Framework are free, but Windows Azure charges by the second for database access. The slower the queries are, the more money Microsoft makes.
So I'm sure Microsoft are working really, really hard to speed it up for you.
If you need speed but cannot get it from EF, there are options:
Write a SQL stored procedure or SQL view - both can be called from Entity Framework.
Write your own query in SQL and execute it using ADO.NET
Fiddle around with the LINQ query until it speeds up by itself

query = query.Where(m => m.Student.StudentEnrollments.Any(s => schoolIds.Contains(s.SchoolId.Value)));
to
query = query.Where(a => schoolIds.Any(b => a.Student.StudentEnrollments.Select(c => c.SchoolId.Value).Contains(b)));
I flipped the logic and it generates a query that increased the performance. Even though it's longer and not ideal, but at least it is "correct". The first LINQ just for some reason has those 2 duplicated blocks which really kills the performance in this case.

How can we create a dynamic sql query to run without considering if the parameters are passed or not in c# windows forms

I am using VS 2012 and SQL Express
I am trying to build a windows forms application to search through a database in C# and it has different controls on the form which are passed as parameters to the query.
The parameters in the query are not necessarily passed some times
I am trying with the following code sample.
SELECT a.ID AS 'DealID', a.TradeDate, c.COMPANYNAME AS 'Seller Company',
a.SellCommission, h.BROKER_FULLNAME AS 'Seller Trader',
j.DisplayName AS 'Seller Broker', d.COMPANYNAME AS 'Buyer Company',
a.BuyCommission, g.BROKER_FULLNAME AS 'Buyer Trader',
i.DisplayName AS 'Buyer Broker', e.PRODUCT_NAME, f.TYPE_DESC AS 'Quantity Type',
f.NBR_OF_GALLONS AS 'Quantity Multiplier', a.ContractVolume, a.TotalVolume,
a.DeliveryPoint, a.Price, a.ContractStart, a.ContractEnd
FROM Confirmations AS a WITH (nolock)
LEFT OUTER JOIN COMPANY AS c WITH (nolock)
ON c.COMPANY_ID = a.SellCompany
LEFT OUTER JOIN COMPANY AS d WITH (nolock)
ON d.COMPANY_ID = a.BuyCompany
LEFT OUTER JOIN BIOPRODUCTTYPES AS e WITH (nolock)
ON e.ID = a.ProductID
LEFT OUTER JOIN BIO_QUANTITY_TYPE AS f WITH (nolock)
ON f.ID = a.QuantityTypeID
LEFT OUTER JOIN COMPANYBROKER AS g WITH (nolock)
ON g.COMPANYBROKER_ID = a.BuyTrader
LEFT OUTER JOIN COMPANYBROKER AS h WITH (nolock)
ON h.COMPANYBROKER_ID = a.SellTrader
LEFT OUTER JOIN Users AS i WITH (nolock)
ON i.ID = a.BuyBroker
LEFT OUTER JOIN Users AS j WITH (nolock)
ON j.ID = a.SellBroker
WHERE (#fromdate IS NULL OR #fromdate=' ' OR a.TradeDate >= #fromdate)
AND (#todate IS NULL OR #todate=' ' OR a.TradeDate <= #todate)
AND (#buycompanyname IS NULL
OR #buycompanyname=""
OR a.BuyCompany = (SELECT COMPANY_ID
FROM COMPANY
WHERE (COMPANYNAME = #buycompanyname)))
AND (#sellcompanyname IS NULL
OR #sellcompanyname=""
OR a.SellCompany = (SELECT COMPANY_ID
FROM COMPANY
WHERE (COMPANYNAME =#sellcompanyname)))
AND (#product IS NULL OR #product="" OR e.PRODUCT_NAME= #product)";
Rather than using the above query, can I dynamically create a query, based on the parameters I passed which seems more logical as it doesn't look for the records if the column in the table has a null value.

This is what ORM's where created for. By replacing your hard coded querys with somthing that builds your query at runtime (like Entity Framework or NHibernate) and it builds the both the SELECT and the WHERE portions of the query for you.
With proper set up objects you could use Entity Framework like the following
Nullable<DateTime> fromDate = //...
Nullable<DateTime> toDate = //...
string buyCompany = //...
//(Snip)
using(var ctx = new MyContext())
{
var query = ctx.Order;
if(fromDate.HasValue)
query = query.Where(ent=> ent.TradeDate >= fromDate.Value);
if(toDate.HasValue)
query = query.Where(ent => ent.TradeDate <= toDate.Value);
if(String.IsNullOrWhitespace(buyCompany) == false)
query = query.Where(ent => ent.BuyCompany.CompanyName = buyCompany);
//(Snip)
return query.ToList();
}

If you are calling a stored procedure, I'd suggest dynamically building the SQL string to only use the parameters you'll be using, then calling sp_executesql. The stored procedure would look like this:
DECLARE #sql =nvarchar(MAX), #Parameters nvarchar(max)
SET #sql = 'SELECT * FROM [dbo].[Foo] WHERE Column1 = #Param1'
SET #Parameters = '#Param1 nvarchar(32), #Param2 nvarchar(32)'
IF(#Param2 is not null and #Param2 <> ' ') SET #sql = #sql + ' AND Column2 = #Param2'
EXEC sp_executesql #Sql, #Parameters, #Param1, #Param2
The idea is basically the same if you're building the query string in C# instead of a stored procedure:
command.CommandText = "SELECT * FROM [dbo].[Foo] WHERE Column1 = #Param1";
command.Parameters.AddWithValue("#Param1", param1);
if(!String.IsNullOrEmpty(param2))
{
command.CommandText += " AND Column2 = #Param2";
command.Parameters.AddWithValue("#Param2", param2);
}

Yes, just build your sql query string out.
Set your SqlCommand.CommandType to CommandType.Text
then set your parameters with SqlCommand.Parameters.AddWidthValue(string, object);