I'm using Dapper to map my database content to EmployeeModel objects.
The mapping of properties is working, but the grouping at the end is still giving me trouble:
EmployeeModel takes List<PhoneModel> and List<EmployeeModel> as properties.
Items are grouped according to EmployeeID, however multiple email and phone results are returned as I haven't been able to find the syntax for doing so.
I've tried looping through the EmployeeIDs in the employeeList after it's been grouped by ID, before it's been grouped by ID, and while it's being grouped by ID.
var sql = #"
SELECT
e.id,
e.FirstName, e.LastName, e.Nickname,
em.id as ID, em.Address, em.Type,
jt.id as ID, jt.Name,
e.id as ID, p.Number, p.Type,
d.id as ID, d.Name,
es.id as ID, es.Name
FROM
dbo.Employees e
LEFT JOIN dbo.Emails em ON em.EmployeeID = e.id
LEFT JOIN dbo.JobTitles jt ON e.JobTitleID = jt.id
LEFT JOIN Phones p ON p.EmployeeID = e.id
LEFT JOIN dbo.Departments d ON e.DepartmentID = d.id
LEFT JOIN dbo.EmployeeStatus es ON e.StatusID = es.id
";
IEnumerable<EmailModel> emailsGrouped = new List<EmailModel>();
var employees = await connection
.QueryAsync<
EmployeeModel,EmailModel,TitleModel,
PhoneModel,DepartmentModel,StatusModel,
EmployeeModel>
(
sql,
( e, em, t, p, d, s ) =>
{
e.EmailList.Add(em);
e.JobTitle = t;
e.PhoneList.Add(p);
e.Department = d;
e.Status = s;
return e;
},
splitOn: "ID, ID, ID, ID, ID"
);
foreach (EmployeeModel emod in employees)
{
emod.EmailList.GroupBy(em => em.ID);
}
var result = employees
.GroupBy(e => e.ID)
.Select(g =>
{
var groupedEmployee = g.First();
groupedEmployee.EmailList = g.Select(e => e.EmailList.Single()).ToList();
groupedEmployee.PhoneList = g.Select(e => e.PhoneList.Single()).ToList();
return groupedEmployee;
});
return result.ToList();
Here is my Email definition, as requested. It's inside my EmployeeClass, so I've posted the whole thing.
public class EmployeeModel
{
public int ID { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
public string Nickname { get; set; }
public DepartmentModel Department { get; set; }
public TitleModel JobTitle { get; set; }
public DateTime HireDate { get; set; }
public StatusModel Status { get; set; }
public List<EmailModel> EmailList { get; set; } = new List<EmailModel>();
public List<PhoneModel> PhoneList { get; set; } = new List<PhoneModel>();
public List<RestrictionModel> RestrictionsList { get; set; } = new List<RestrictionModel>();
public List<CitationModel> CitationsList { get; set; } = new List<CitationModel>();
public List<CertificationModel> CertificationList { get; set; } = new List<CertificationModel>();
public string ListView
{
get
{
return $"{LastName}, {FirstName}";
}
}
public string ToEmailString()
{
IEnumerable<string> employeeEmailStrings = EmailList.Select(emmod => emmod.ToString());
string employeeEmailString = string.Join($"{Environment.NewLine}", employeeEmailStrings);
return $"{FirstName}, {LastName}: {Environment.NewLine} -{JobTitle.Name}- {Environment.NewLine}";
}
//IEnumerable<string> phoneStrings = PhoneList.Select(plistmod => plistmod.ToString());
//string phoneString = string.Join($"{Environment.NewLine}", phoneStrings);
public string ToCertificationString()
{
IEnumerable<string> certificationStrings = CertificationList.Select(clistmod => clistmod.ToString());
string certificationString = string.Join($"{Environment.NewLine}", certificationStrings);
return certificationString;
}
public class EmailModel
{
public int ID { get; set; }
public string Address { get; set; }
public string Type { get; set; }
public override string ToString()
{
return $"{Address} ({Type})";
}
}
public class PhoneModel
{
public int ID { get; set; }
public string Number { get; set; }
public string Type { get; set; }
public override string ToString()
{
return $"{Number} ({Type})";
}
}
}
}
What I'm trying now is to loop through the Emails in the EmployeeModel to create a new list of emails, and then set that new list as the EmployeeModel.List<EmailModel>.
So it looks like you're actually trying to load an object-graph (containing nodes of distinct types) from a database using SQL - and you're trying to do that using a single query.
That won't work. (Naïve, single-query) SQL is not suitable for querying object-graphs. This is why ORMs exist. However with some RDBMS-specific SQL extensions (e.g. T-SQL, PL/SQL, etc) to execute a query batch you can return an object-graph from a database.
The good news is that Dapper supports this scenario with QueryMultiple - however as far as I know it won't map collection properties, so you need to do that manually (so read on!)
(I note that Entity Framework, specifically, will generate single-SELECT queries that return redundant data in columns that represent lower-multiplicity data - this has its trade-offs but generally speaking separate queries can work faster overall with the right tweaks (such as using a table-valued variable to hold KEY values instead of re-evaluating the same WHERE criteria for every query in the batch - as always, check your indexes, STATISTICS objects, and execution plans!).
When querying for an object-graph, you'll write a SELECT query batch where each query returns all objects of the same type that has a JOIN with any other entities with a 1:1 or 1:0..1 multiplicity (if it isn't more efficient to load them in a separate query in the same batch).
In your case, I see you have:
[Employees]---(1:m)---[Phones]
[Employees]---(1:m)---[Emails]
[JobTitles]---(1:m)---[Employees]
[Departments]---(1:m)---[Employees]
[EmployeeStatus]---(1:m)---[Employees] // is this an enum table? if so, you can probably ditch it
So try this:
For the sake of simplicity, JobTitles, Departments, and EmployeeStatus can be done in a single query.
I assume the foreign-key columns are NOT NULL so an INNER JOIN should be used instead of LEFT OUTER JOIN.
const String EMPLOYEES_PHONES_EMAILS_SQL = #"
-- Query 1: Employees, Departments, EmployeeStatuses
SELECT
e.id,
e.FirstName,
e.LastName,
e.Nickname,
t.Name AS JobTitleName, -- This is to disambiguate column names. Never rely on column ordinals!
d.Name AS DepartmentName,
s.Name AS StatusName
FROM
dbo.Employees AS e
INNER JOIN dbo.JobTitles AS t ON e.JobTitleID = t.id
INNER JOIN dbo.Departments AS d ON e.DepartmentId = d.id
INNER JOIN dbo.EmployeeStatus AS s ON e.StatusID = s.id;
-- Query 2: Phones
SELECT
p.EmployeeId,
p.Number,
p.Type
FROM
dbo.Phones AS p;
-- Query 3: Emails
SELECT
m.id,
m.EmployeeId,
m.Address,
m.Type
FROM
dbo.Emails AS m;
";
using( SqlMapper.GridReader rdr = connection.QueryMultiple( EMPLOYEES_PHONES_EMAILS_SQL ) )
{
List<EmployeeModel> employees = ( await rdr.ReadAsync<EmployeeModel>() ).ToList();
var phonesByEmployeeId = ( await rdr.ReadAsync<PhoneModel> () ).GroupBy( p => p.EmployeeId ).Dictionary( grp => grp.Key grp => grp.ToList() );
var emailsByEmployeeId = ( await rdr.ReadAsync<EmailModel> () ).GroupBy( m => m.EmployeeId ).Dictionary( grp => grp.Key, grp => grp.ToList() );
foreach( EmployeeModel emp in employees )
{
if( phonesByEmployeeId.TryGetValue( emp.EmployeeId, out var phones ) )
{
emp.Phones.AddRange( phones );
}
if( emailsByEmployeeId.TryGetValue( emp.EmployeeId, out var emails ) )
{
emp.Emails.AddRange( emails );
}
}
}
I'll admit that I'm not intimately familiar with Dapper - and there is a problem with the code above: it doesn't instruct Dapper how to read the included Department, JobTitleModel, and EmployeeStatus data in the first query. I assume there's some overload of ReadAsync to specify other included data.
If you find yourself doing this kind of logic repetitively you can define your own extension-methods to handle the worst parts (such as GroupBy().ToDictionary(), and populating a collection property from a dictionary of loaded entities).
If you had a filter criteria, then you'd need to either store the resultant EmployeeId key values in a TVV, or repeat the criteria on Employees as the right-hand-side of an INNER JOIN in the queries for Phones and Emails.
For example, if you wanted to add an ability to find all Employees (and their phone-numbers and e-mail addresses) by name, you'd do this:
const String EMPLOYEES_PHONES_EMAILS_SQL = #"
-- Query 0: Get EmployeeIds:
DECLARE #empIds TABLE ( EmpId int NOT NULL PRIMARY KEY );
INSERT INTO #empIds ( EmpId )
SELECT
EmployeeId
FROM
dbo.Employees
WHERE
FirstName LIKE #likeFirst
OR
LastName LIKE #likeLast;
-- Query 1: Employees, Departments, EmployeeStatuses
SELECT
e.id,
e.FirstName,
e.LastName,
e.Nickname,
t.Name AS JobTitleName, -- This is to disambiguate column names. Never rely on column ordinals!
d.Name AS DepartmentName,
s.Name AS StatusName
FROM
dbo.Employees AS e
INNER JOIN dbo.JobTitles AS t ON e.JobTitleID = t.id
INNER JOIN dbo.Departments AS d ON e.DepartmentId = d.id
INNER JOIN dbo.EmployeeStatus AS s ON e.StatusID = s.id
INNER JOIN #empIds AS i ON i.EmpId = e.EmployeeId;
-- Query 2: Phones
SELECT
p.EmployeeId,
p.Number,
p.Type
FROM
dbo.Phones AS p
INNER JOIN #empIds AS i ON i.EmpId = p.EmployeeId;
-- Query 3: Emails
SELECT
m.id,
m.EmployeeId,
m.Address,
m.Type
FROM
dbo.Emails AS m
INNER JOIN #empIds AS i ON i.EmpId = m.EmployeeId;
";
using( SqlMapper.GridReader rdr = connection.QueryMultiple( EMPLOYEES_PHONES_EMAILS_SQL, new { likeFirst = "%john%", likeLast = "%smith%" } ) )
{
// same as before
}
I need to query two database tables for a search term and return the results. The following was working in EF Core 2:
var SearchTerm = "hello";
IQueryable<TableA> q;
q = (from a in context.TableA
join b in context.TableB on a equals b.A into leftjoin
from c in leftjoin.DefaultIfEmpty()
where c.Column1.Contains(SearchTerm)
|| a.Column1.Contains(SearchTerm)
|| a.Column2.Contains(SearchTerm)
select a);
return q.Include(a => a.TableD)
.GroupBy(a => a.Id)
.Select(group => group.First())
.ToList();
The idea of the above is to take a SearchTerm and query two columns from TableA, join to TableB and also query a column in this one then select distinct values from TableA.
In .NET 3 the above throws an error saying it can't be translated to SQL. I tried to rewrite this, the best I can do is the below:
var SearchTerm = "hello";
var q = (from a in context.TableA
join b in context.TableB on a equals b.A into leftjoin
from c in leftjoin.DefaultIfEmpty()
where c.Column1.Contains(SearchTerm)
|| a.Column1.Contains(SearchTerm)
|| a.Column2.Contains(SearchTerm)
select a.Id).Distinct().ToList();
return context.TableA
.Where(a => q.Contains(a.Id))
.Include(c => c.TableD)
.ToList();
Which works ok but involves two database queries, since I already have the list of TableA from the first query it would be great to be able to just use this without having to extract the Ids and performing the second query. Also making sure the database continues to handle the distinct part rather than C# would be preferable too.
The definitions of A and B are:
public class TableA
{
public int Id { get; set; }
public string Column1 { get; set; }
public string Column2 { get; set; }
public int TableDId { get; set; }
public TableD TableD { get; set; }
}
public class TableB
{
public int Id { get; set; }
public string Column1 { get; set; }
public int TableAId { get; set; }
public TableA TableA { get; set; }
}
If I understood you correctly you have one-to-many relation between TableA and TableB, so it should be possible to add collection navigation property to TableA like in this tutorial for example:
public class TableA
{
public int Id { get; set; }
...
public ICollection<TableB> TableBs { get; set; }
}
So you can try to do something like this:
context.TableA
.Where(ta => ta.TableBs.Any(tb => tb.Column1.Contains(SearchTerm))
|| ta.Column1.Contains(SearchTerm)
|| ta.Column2.Contains(SearchTerm))
.Include(c => c.TableD)
.ToList();
Another option is to try subquery:
var q = (from a in context.TableA
join b in context.TableB on a.Id equals b.TableAId into leftjoin
from c in leftjoin.DefaultIfEmpty()
where c.Column1.Contains(SearchTerm)
|| a.Column1.Contains(SearchTerm)
|| a.Column2.Contains(SearchTerm)
select a.Id); // remove Distinct and ToList
return context.TableA
.Where(a => q.Contains(a.Id))
.Include(c => c.TableD)
.ToList();
I'm trying to convert the following Join statement into LINQ TO SQL or LINQ to Entity. I know how to join tables in either implementation; but, i'm struggling with the AND clause in the Join statement.
SELECT DISTINCT
p.LastName,
p.FirstName
FROM
dbo.Patient p INNER JOIN dbo.FormPat fp ON p.PatientID = fp.PatientID
INNER JOIN dbo.TxCyclePhase tcp ON fp.TxCyclePhase = tcp.TxCyclePhaseID AND tcp.Type = 2
As far as LINQ to SQL is concerned, I have the followings:
var query = (from p in Context.Set<Patient>().AsNoTracking()
join fp in Context.Set<PatientForm>().AsNoTracking() on p.Id equals fp.PatientId
join tcp in Context.Set<TxCyclePhase>().AsNoTracking() on new { fp.TxCyclePhaseId, seconProperty = true } equals new { tcp.Id, seconProperty = tcp.Type == 2 }
select new
{
p.FirstName,
p.LastName,
}).Distinct();
However, I'm getting an ArgumentNullException on the second join statement.
For the LINQ to Entity, I have the followings, however, this is giving me a distinct IQueryable of FormPat, instead of Patient.
var patients = Context.Set<Patient>().AsNoTracking()
.SelectMany(p => p.Forms)
.Where(fp => fp.Phase.Type == 2)
.Distinct();
As far as the LINQ to Entity is concerned, I was able to figure it out. I'd still like to know how to do it in LINQ to SQL tho.
I'm using the EF fluent API. My Patient object looks like:
public Patient()
{
Programs = new HashSet<Program>();
}
public virtual ICollection<PatientForm> Forms { get; set; }
My PatientForm object looks like:
public class PatientForm
{
public int FormId { get; set; }
public Patient CurrentPatient { get; set; }
public TxCyclePhase Phase { get; set; }
}
And the CyclePhase object looks like:
public TxCyclePhase()
{
this.FormPats = new HashSet<PatientForm>();
}
public int Id { get; set; }
public virtual ICollection<PatientForm> FormPats { get; set; }
In the entity configurations, I have the relationships set. So, in the repository, all I have to do is to use the Any() function when selecting the Patient forms.
var patients = Context.Set<Patient>().AsNoTracking()
.Where(p => p.Forms.Any(f => f.Phase.Type == 2))
.Distinct();
I am using linq2db as ORM for my project (ASP.NET Core 2.2) with Oracle database. I want to get list of products from database ordered by a specific column but this column doesn't need to be shown on the front-end client.
SQL query that I am looking for should be something like this:
SELECT Id, Name
FROM Products
ORDER BY InternalOrder
So when I use linq2db in my C# code like this:
using (var db = new ProductsDao())
{
return db.Products.Select(p => new Product
{
Id = p.Id,
MarketName = p.MarketName
})
.OrderBy(p => p.InternalOrder)
.ToArray();
}
then linq2db converts this to the following SQL query:
SELECT
t1.Id,
t1.Name
FROM
Products t1
ORDER BY
NULL
Notice the "ORDER BY NULL" which effectively won't do any ordering.
However if I add InternalOrder in the Select() like this:
using (var db = new ProductsDao())
{
return db.Products.Select(p => new Product
{
Id = p.Id,
MarketName = p.MarketName,
InternalOrder = p.InternalOrder // this is added
})
.OrderBy(p => p.InternalOrder)
.ToArray();
}
then I get correct ORDER BY clause in the generated SQL query:
SELECT
t1.Id,
t1.Name,
t1.InternalOrder
FROM
Products t1
ORDER BY
t1.InternalOrder
However, in this case I had to include InternalOrder column in the SELECT statement which is something I want to avoid because I don't need that field in the results.
Note, in the real application I might have very big tables and with multiple columns that I want to sort by but don't want to include in the result. Not getting values from database for these columns should save me some performance costs.
For code completeness here are my linq2db related classes that I used in the example above:
[Table(Name = "Products")]
public class Product
{
[Column(Name = "Id")]
public decimal Id { get; set; }
[Column(Name = "Name")]
public string Name { get; set; }
[Column(Name = "InternalOrder", CanBeNull = false)]
public int InternalOrder { get; set; }
}
public class LiveEventServiceDao : LinqToDB.Data.DataConnection
{
public LiveEventServiceDao() : base("MyOracleDb")
{
}
public ITable<Product> Products => GetTable<Product>();
}
Is there a way to accomplish this with linq2db?
Try following :
return db.Products.Select(p => new {Order = p.InternalOrder, Prod = p})
.OrderBy(p => p.Order)
.Select(p => new Product {
{
Id = p.Prod.Id,
MarketName = p.Prod.MarketName
})
.ToArray();
I have two tables A and B. The domain object pulls most of its data from A, and some aggregation from B.
For example:
Table A ( id, name );
Table B ( id_A, quantity );
class A {
public int id { set; get; }
public string name { set; get; }
}
class B {
public int id_A { set; get; }
public int quantity { set; get; }
}
var result =
from a in A join b in B on a.id equals b.id_A
group b by b.id_A into g
select new {
Name = a.name,
Total = g.Sum( b => b.quantity )
};
Instead of creating an anonymous type, I'd like to add a property to domain object A called it TotalQuantity and populate it with g.Sum( b => b.quantity ). I'd also like to turn result into IEnumerable instead of var.
My first bet was
class A {
public int id { set; get; }
public string name { set; get; }
public int TotalQuantity { set; get; }
}
IEnumerable<A> result =
from a in A join b in B on a.id equals b.id_A
group b by b.id_A into g
select new A {
name = a.name,
TotalQuantity = g.Sum( b => b.quantity )
};
This operation is not supported by the runtime:
System.NotSupportedException: Explicit construction of entity type 'Data.A' in query is not allowed.
Note that domain A and B doesn't contain any reference to each other. Their relationship is not used explicitly in the application, therefore, I chose not to model it.
How can I neatly populate a list of A without looping through the data stored in the instances of the anonymous class?
This should do it (note I have not tested it so some tweaking may be in order):
IEnumerable <A> result =
(from a in A join b in B on a.id equals b.id_A
group b by b.id_A into g
select new {
Name = a.name,
Total = g.Sum( b => b.quantity )
}).Select(obj => new A {Name = obj.Name, TotalQuantity = obj.Total});
You'll have perform your projection in memory instead of the database. This way the LINQ to SQL provider won't attempt to convert it to an SQL query.
Here's an example:
IEnumerable<A> result = (from a in A join b in B on a.id equals b.id_A
group b by b.id_A into g
select new
{
Name = a.name,
Total = g.Sum(b => b.quantity)
})
.ToArray()
.Select(item => new A
{
Name = item.Name,
TotalQuantity = item.Total
});
The call to the IQueryable<T>.ToArray() method will force the LINQ to SQL provider to run the query against the database and return the results in an array. The final projection is then performed in memory, circumventing the limitations of the LINQ to SQL provider.
Related resources:
LINQ and Deferred Execution
The performance implications of IEnumerable vs. IQueryable