I'm using Dapper to map my database content to EmployeeModel objects.
The mapping of properties is working, but the grouping at the end is still giving me trouble:
EmployeeModel takes List<PhoneModel> and List<EmployeeModel> as properties.
Items are grouped according to EmployeeID, however multiple email and phone results are returned as I haven't been able to find the syntax for doing so.
I've tried looping through the EmployeeIDs in the employeeList after it's been grouped by ID, before it's been grouped by ID, and while it's being grouped by ID.
var sql = #"
SELECT
e.id,
e.FirstName, e.LastName, e.Nickname,
em.id as ID, em.Address, em.Type,
jt.id as ID, jt.Name,
e.id as ID, p.Number, p.Type,
d.id as ID, d.Name,
es.id as ID, es.Name
FROM
dbo.Employees e
LEFT JOIN dbo.Emails em ON em.EmployeeID = e.id
LEFT JOIN dbo.JobTitles jt ON e.JobTitleID = jt.id
LEFT JOIN Phones p ON p.EmployeeID = e.id
LEFT JOIN dbo.Departments d ON e.DepartmentID = d.id
LEFT JOIN dbo.EmployeeStatus es ON e.StatusID = es.id
";
IEnumerable<EmailModel> emailsGrouped = new List<EmailModel>();
var employees = await connection
.QueryAsync<
EmployeeModel,EmailModel,TitleModel,
PhoneModel,DepartmentModel,StatusModel,
EmployeeModel>
(
sql,
( e, em, t, p, d, s ) =>
{
e.EmailList.Add(em);
e.JobTitle = t;
e.PhoneList.Add(p);
e.Department = d;
e.Status = s;
return e;
},
splitOn: "ID, ID, ID, ID, ID"
);
foreach (EmployeeModel emod in employees)
{
emod.EmailList.GroupBy(em => em.ID);
}
var result = employees
.GroupBy(e => e.ID)
.Select(g =>
{
var groupedEmployee = g.First();
groupedEmployee.EmailList = g.Select(e => e.EmailList.Single()).ToList();
groupedEmployee.PhoneList = g.Select(e => e.PhoneList.Single()).ToList();
return groupedEmployee;
});
return result.ToList();
Here is my Email definition, as requested. It's inside my EmployeeClass, so I've posted the whole thing.
public class EmployeeModel
{
public int ID { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
public string Nickname { get; set; }
public DepartmentModel Department { get; set; }
public TitleModel JobTitle { get; set; }
public DateTime HireDate { get; set; }
public StatusModel Status { get; set; }
public List<EmailModel> EmailList { get; set; } = new List<EmailModel>();
public List<PhoneModel> PhoneList { get; set; } = new List<PhoneModel>();
public List<RestrictionModel> RestrictionsList { get; set; } = new List<RestrictionModel>();
public List<CitationModel> CitationsList { get; set; } = new List<CitationModel>();
public List<CertificationModel> CertificationList { get; set; } = new List<CertificationModel>();
public string ListView
{
get
{
return $"{LastName}, {FirstName}";
}
}
public string ToEmailString()
{
IEnumerable<string> employeeEmailStrings = EmailList.Select(emmod => emmod.ToString());
string employeeEmailString = string.Join($"{Environment.NewLine}", employeeEmailStrings);
return $"{FirstName}, {LastName}: {Environment.NewLine} -{JobTitle.Name}- {Environment.NewLine}";
}
//IEnumerable<string> phoneStrings = PhoneList.Select(plistmod => plistmod.ToString());
//string phoneString = string.Join($"{Environment.NewLine}", phoneStrings);
public string ToCertificationString()
{
IEnumerable<string> certificationStrings = CertificationList.Select(clistmod => clistmod.ToString());
string certificationString = string.Join($"{Environment.NewLine}", certificationStrings);
return certificationString;
}
public class EmailModel
{
public int ID { get; set; }
public string Address { get; set; }
public string Type { get; set; }
public override string ToString()
{
return $"{Address} ({Type})";
}
}
public class PhoneModel
{
public int ID { get; set; }
public string Number { get; set; }
public string Type { get; set; }
public override string ToString()
{
return $"{Number} ({Type})";
}
}
}
}
What I'm trying now is to loop through the Emails in the EmployeeModel to create a new list of emails, and then set that new list as the EmployeeModel.List<EmailModel>.
So it looks like you're actually trying to load an object-graph (containing nodes of distinct types) from a database using SQL - and you're trying to do that using a single query.
That won't work. (Naïve, single-query) SQL is not suitable for querying object-graphs. This is why ORMs exist. However with some RDBMS-specific SQL extensions (e.g. T-SQL, PL/SQL, etc) to execute a query batch you can return an object-graph from a database.
The good news is that Dapper supports this scenario with QueryMultiple - however as far as I know it won't map collection properties, so you need to do that manually (so read on!)
(I note that Entity Framework, specifically, will generate single-SELECT queries that return redundant data in columns that represent lower-multiplicity data - this has its trade-offs but generally speaking separate queries can work faster overall with the right tweaks (such as using a table-valued variable to hold KEY values instead of re-evaluating the same WHERE criteria for every query in the batch - as always, check your indexes, STATISTICS objects, and execution plans!).
When querying for an object-graph, you'll write a SELECT query batch where each query returns all objects of the same type that has a JOIN with any other entities with a 1:1 or 1:0..1 multiplicity (if it isn't more efficient to load them in a separate query in the same batch).
In your case, I see you have:
[Employees]---(1:m)---[Phones]
[Employees]---(1:m)---[Emails]
[JobTitles]---(1:m)---[Employees]
[Departments]---(1:m)---[Employees]
[EmployeeStatus]---(1:m)---[Employees] // is this an enum table? if so, you can probably ditch it
So try this:
For the sake of simplicity, JobTitles, Departments, and EmployeeStatus can be done in a single query.
I assume the foreign-key columns are NOT NULL so an INNER JOIN should be used instead of LEFT OUTER JOIN.
const String EMPLOYEES_PHONES_EMAILS_SQL = #"
-- Query 1: Employees, Departments, EmployeeStatuses
SELECT
e.id,
e.FirstName,
e.LastName,
e.Nickname,
t.Name AS JobTitleName, -- This is to disambiguate column names. Never rely on column ordinals!
d.Name AS DepartmentName,
s.Name AS StatusName
FROM
dbo.Employees AS e
INNER JOIN dbo.JobTitles AS t ON e.JobTitleID = t.id
INNER JOIN dbo.Departments AS d ON e.DepartmentId = d.id
INNER JOIN dbo.EmployeeStatus AS s ON e.StatusID = s.id;
-- Query 2: Phones
SELECT
p.EmployeeId,
p.Number,
p.Type
FROM
dbo.Phones AS p;
-- Query 3: Emails
SELECT
m.id,
m.EmployeeId,
m.Address,
m.Type
FROM
dbo.Emails AS m;
";
using( SqlMapper.GridReader rdr = connection.QueryMultiple( EMPLOYEES_PHONES_EMAILS_SQL ) )
{
List<EmployeeModel> employees = ( await rdr.ReadAsync<EmployeeModel>() ).ToList();
var phonesByEmployeeId = ( await rdr.ReadAsync<PhoneModel> () ).GroupBy( p => p.EmployeeId ).Dictionary( grp => grp.Key grp => grp.ToList() );
var emailsByEmployeeId = ( await rdr.ReadAsync<EmailModel> () ).GroupBy( m => m.EmployeeId ).Dictionary( grp => grp.Key, grp => grp.ToList() );
foreach( EmployeeModel emp in employees )
{
if( phonesByEmployeeId.TryGetValue( emp.EmployeeId, out var phones ) )
{
emp.Phones.AddRange( phones );
}
if( emailsByEmployeeId.TryGetValue( emp.EmployeeId, out var emails ) )
{
emp.Emails.AddRange( emails );
}
}
}
I'll admit that I'm not intimately familiar with Dapper - and there is a problem with the code above: it doesn't instruct Dapper how to read the included Department, JobTitleModel, and EmployeeStatus data in the first query. I assume there's some overload of ReadAsync to specify other included data.
If you find yourself doing this kind of logic repetitively you can define your own extension-methods to handle the worst parts (such as GroupBy().ToDictionary(), and populating a collection property from a dictionary of loaded entities).
If you had a filter criteria, then you'd need to either store the resultant EmployeeId key values in a TVV, or repeat the criteria on Employees as the right-hand-side of an INNER JOIN in the queries for Phones and Emails.
For example, if you wanted to add an ability to find all Employees (and their phone-numbers and e-mail addresses) by name, you'd do this:
const String EMPLOYEES_PHONES_EMAILS_SQL = #"
-- Query 0: Get EmployeeIds:
DECLARE #empIds TABLE ( EmpId int NOT NULL PRIMARY KEY );
INSERT INTO #empIds ( EmpId )
SELECT
EmployeeId
FROM
dbo.Employees
WHERE
FirstName LIKE #likeFirst
OR
LastName LIKE #likeLast;
-- Query 1: Employees, Departments, EmployeeStatuses
SELECT
e.id,
e.FirstName,
e.LastName,
e.Nickname,
t.Name AS JobTitleName, -- This is to disambiguate column names. Never rely on column ordinals!
d.Name AS DepartmentName,
s.Name AS StatusName
FROM
dbo.Employees AS e
INNER JOIN dbo.JobTitles AS t ON e.JobTitleID = t.id
INNER JOIN dbo.Departments AS d ON e.DepartmentId = d.id
INNER JOIN dbo.EmployeeStatus AS s ON e.StatusID = s.id
INNER JOIN #empIds AS i ON i.EmpId = e.EmployeeId;
-- Query 2: Phones
SELECT
p.EmployeeId,
p.Number,
p.Type
FROM
dbo.Phones AS p
INNER JOIN #empIds AS i ON i.EmpId = p.EmployeeId;
-- Query 3: Emails
SELECT
m.id,
m.EmployeeId,
m.Address,
m.Type
FROM
dbo.Emails AS m
INNER JOIN #empIds AS i ON i.EmpId = m.EmployeeId;
";
using( SqlMapper.GridReader rdr = connection.QueryMultiple( EMPLOYEES_PHONES_EMAILS_SQL, new { likeFirst = "%john%", likeLast = "%smith%" } ) )
{
// same as before
}
Related
I need to query two database tables for a search term and return the results. The following was working in EF Core 2:
var SearchTerm = "hello";
IQueryable<TableA> q;
q = (from a in context.TableA
join b in context.TableB on a equals b.A into leftjoin
from c in leftjoin.DefaultIfEmpty()
where c.Column1.Contains(SearchTerm)
|| a.Column1.Contains(SearchTerm)
|| a.Column2.Contains(SearchTerm)
select a);
return q.Include(a => a.TableD)
.GroupBy(a => a.Id)
.Select(group => group.First())
.ToList();
The idea of the above is to take a SearchTerm and query two columns from TableA, join to TableB and also query a column in this one then select distinct values from TableA.
In .NET 3 the above throws an error saying it can't be translated to SQL. I tried to rewrite this, the best I can do is the below:
var SearchTerm = "hello";
var q = (from a in context.TableA
join b in context.TableB on a equals b.A into leftjoin
from c in leftjoin.DefaultIfEmpty()
where c.Column1.Contains(SearchTerm)
|| a.Column1.Contains(SearchTerm)
|| a.Column2.Contains(SearchTerm)
select a.Id).Distinct().ToList();
return context.TableA
.Where(a => q.Contains(a.Id))
.Include(c => c.TableD)
.ToList();
Which works ok but involves two database queries, since I already have the list of TableA from the first query it would be great to be able to just use this without having to extract the Ids and performing the second query. Also making sure the database continues to handle the distinct part rather than C# would be preferable too.
The definitions of A and B are:
public class TableA
{
public int Id { get; set; }
public string Column1 { get; set; }
public string Column2 { get; set; }
public int TableDId { get; set; }
public TableD TableD { get; set; }
}
public class TableB
{
public int Id { get; set; }
public string Column1 { get; set; }
public int TableAId { get; set; }
public TableA TableA { get; set; }
}
If I understood you correctly you have one-to-many relation between TableA and TableB, so it should be possible to add collection navigation property to TableA like in this tutorial for example:
public class TableA
{
public int Id { get; set; }
...
public ICollection<TableB> TableBs { get; set; }
}
So you can try to do something like this:
context.TableA
.Where(ta => ta.TableBs.Any(tb => tb.Column1.Contains(SearchTerm))
|| ta.Column1.Contains(SearchTerm)
|| ta.Column2.Contains(SearchTerm))
.Include(c => c.TableD)
.ToList();
Another option is to try subquery:
var q = (from a in context.TableA
join b in context.TableB on a.Id equals b.TableAId into leftjoin
from c in leftjoin.DefaultIfEmpty()
where c.Column1.Contains(SearchTerm)
|| a.Column1.Contains(SearchTerm)
|| a.Column2.Contains(SearchTerm)
select a.Id); // remove Distinct and ToList
return context.TableA
.Where(a => q.Contains(a.Id))
.Include(c => c.TableD)
.ToList();
I have a nested data model and I'd like to get aggregated data from it grouped by a top level property.
My models for example:
public class Scan {
public long Id {get; set;}
public int ProjectId { get; set; }
public int ScanUserId { get; set; }
public ICollection<ScanItem>? Items { get; set; }
}
public class ScanItem
{
public long Id { get; set; }
public long InventoryScanId { get; set; }
public double Qty { get; set; }
}
I'd like to get all Scans grouped by Scan.ScanUserId and then then get the sum of ScanItems.Qty for example per user. My query looks ike this and EF gives the following error:
Processing of the LINQ expression 'AsQueryable((Unhandled
parameter: x).Items)' by 'NavigationExpandingExpressionVisitor' failed
from scan in Scans
.Include(x=>x.ScanUser)
.Include(x=>x.Items)
group scan by new { scan.ScanUser.Name, scan.ScanUser.Id } into g
select new
{
UserId = g.Key.Id,
Name = g.Key.Name,
LastSyncTime = g.Max(x => x.ScanDate),
ScanItems = g.Sum(x=>x.Items.Sum(i=>i.Qty))
}
How can I run aggregate functions on the properties of the nested table without evaluating it on the client side?
EF Core still can't translate nested aggregates on GroupBy result (grouping).
You have to pre calculate the nested aggregates in advance by utilizing the element selector of GroupBy (or element in query syntax group element by key):
from scan in Scans
group new { scan.ScanDate, Qty = scan.Items.Sum(i => i.Qty) } // <--
by new { scan.ScanUser.Name, scan.ScanUser.Id } into g
select new
{
UserId = g.Key.Id,
Name = g.Key.Name,
LastSyncTime = g.Max(x => x.ScanDate),
ScanItems = g.Sum(x => x.Qty) // <--
}
**Update: ** for SqlServer the above LINQ query translates to SQL query like this:
SELECT [s1].[Id] AS [UserId], [s1].[Name], MAX([s0].[ScanDate]) AS [LastSyncTime], SUM((
SELECT SUM([s].[Qty])
FROM [ScanItem] AS [s]
WHERE [s0].[Id] = [s].[InventoryScanId])) AS [ScanItems]
FROM [Scan] AS [s0]
INNER JOIN [ScanUser] AS [s1] ON [s0].[ScanUserId] = [s1].[Id]
GROUP BY [s1].[Name], [s1].[Id]
which as mentioned in the comment generates SQL execution exception "Cannot perform an aggregate function on an expression containing an aggregate or a subquery.".
So you really need another approach - use left join to flatten the result set before grouping, and then perform the grouping / aggregates on that set:
from scan in Scans
from item in scan.Items.DefaultIfEmpty() // <-- left outer join
group new { scan, item } by new { scan.ScanUser.Name, scan.ScanUser.Id } into g
select new
{
UserId = g.Key.Id,
Name = g.Key.Name,
LastSyncTime = g.Max(x => x.scan.ScanDate),
ScanItems = g.Sum(x => (double?)x.item.Qty) ?? 0
};
which now translates to hopefully valid SqlServer SQL query:
SELECT [s1].[Id] AS [UserId], [s1].[Name], MAX([s].[ScanDate]) AS [LastSyncTime], COALESCE(SUM([s0].[Qty]), 0.0E0) AS [ScanItems]
FROM [Scan] AS [s]
LEFT JOIN [ScanItem] AS [s0] ON [s].[Id] = [s0].[InventoryScanId]
INNER JOIN [ScanUser] AS [s1] ON [s].[ScanUserId] = [s1].[Id]
GROUP BY [s1].[Name], [s1].[Id]
I'm trying to convert the following Join statement into LINQ TO SQL or LINQ to Entity. I know how to join tables in either implementation; but, i'm struggling with the AND clause in the Join statement.
SELECT DISTINCT
p.LastName,
p.FirstName
FROM
dbo.Patient p INNER JOIN dbo.FormPat fp ON p.PatientID = fp.PatientID
INNER JOIN dbo.TxCyclePhase tcp ON fp.TxCyclePhase = tcp.TxCyclePhaseID AND tcp.Type = 2
As far as LINQ to SQL is concerned, I have the followings:
var query = (from p in Context.Set<Patient>().AsNoTracking()
join fp in Context.Set<PatientForm>().AsNoTracking() on p.Id equals fp.PatientId
join tcp in Context.Set<TxCyclePhase>().AsNoTracking() on new { fp.TxCyclePhaseId, seconProperty = true } equals new { tcp.Id, seconProperty = tcp.Type == 2 }
select new
{
p.FirstName,
p.LastName,
}).Distinct();
However, I'm getting an ArgumentNullException on the second join statement.
For the LINQ to Entity, I have the followings, however, this is giving me a distinct IQueryable of FormPat, instead of Patient.
var patients = Context.Set<Patient>().AsNoTracking()
.SelectMany(p => p.Forms)
.Where(fp => fp.Phase.Type == 2)
.Distinct();
As far as the LINQ to Entity is concerned, I was able to figure it out. I'd still like to know how to do it in LINQ to SQL tho.
I'm using the EF fluent API. My Patient object looks like:
public Patient()
{
Programs = new HashSet<Program>();
}
public virtual ICollection<PatientForm> Forms { get; set; }
My PatientForm object looks like:
public class PatientForm
{
public int FormId { get; set; }
public Patient CurrentPatient { get; set; }
public TxCyclePhase Phase { get; set; }
}
And the CyclePhase object looks like:
public TxCyclePhase()
{
this.FormPats = new HashSet<PatientForm>();
}
public int Id { get; set; }
public virtual ICollection<PatientForm> FormPats { get; set; }
In the entity configurations, I have the relationships set. So, in the repository, all I have to do is to use the Any() function when selecting the Patient forms.
var patients = Context.Set<Patient>().AsNoTracking()
.Where(p => p.Forms.Any(f => f.Phase.Type == 2))
.Distinct();
I have the following data model:
public class Course
{
public int CourseId { get; set; }
public int StateId { get; set; }
}
public class CompletedCourse
{
public int CompletedCourseId { get; set; }
public int UserId { get; set; }
public Course Course { get; set; }
public string LicenseNumber { get; set; }
}
public class License
{
public int LicenseId { get; set; }
public int UserId { get; set; }
public int StateId { get; set; }
public string LicenseNumber { get; set; }
}
I'm trying to come up with an IQueryable for CompletedCourses and I would like to populate CompletedCourse.LicenseNumber with the LicenseNumber property of the FirstOrDefault() selection from my Licenses table where UserId and StateId match the completed course records.
Here is my query, but I don't think this will handle duplicate licenses correctly:
var entries =
(from course in context.CompletedCourses
join license in context.Licenses on course.UserId equals license.UserId
where license.StateId == course.Course.StateId
select course)
.Include(x => x.Agent)
.Include(x => x.Course.State);
Is this something that can be done in a single query? Thanks in advance.
Here is how you can do that:
var entries =
(from course in context.CompletedCourses
join license in context.Licenses
on new { course.UserId, course.Course.StateId }
equals new { license.UserId, license.StateId }
into licenses
let licenseNumber = licenses.Select(license => license.LicenseNumber).FirstOrDefault()
select new { course, licenseNumber });
But please note that with this type of projection you cannot have Includes in your query (you can, but they will not be in effect).
The EF generated query I'm getting from the above is:
SELECT
[Extent1].[CompletedCourseId] AS [CompletedCourseId],
[Extent1].[UserId] AS [UserId],
[Extent1].[LicenseNumber] AS [LicenseNumber],
[Extent1].[Course_CourseId] AS [Course_CourseId],
(SELECT TOP (1)
[Extent2].[LicenseNumber] AS [LicenseNumber]
FROM [dbo].[Licenses] AS [Extent2]
INNER JOIN [dbo].[Courses] AS [Extent3] ON [Extent3].[StateId] = [Extent2].[StateId]
WHERE ([Extent1].[Course_CourseId] = [Extent3].[CourseId]) AND ([Extent1].[UserId] = [Extent2].[UserId])) AS [C1]
FROM [dbo].[CompletedCourses] AS [Extent1]
It can be noticed that EF effectively ignores the join, so the same result can be obtained by simple natural query:
var entries =
(from course in db.CompletedCourses
let licenseNumber =
(from license in db.Licenses
where license.UserId == course.UserId && license.StateId == course.Course.StateId
select license.LicenseNumber).FirstOrDefault()
select new { course, licenseNumber });
#IvanStoev's answer was very helpful in joining on anonymous types, but ultimately I couldn't use it because I needed Includes. Here is the solution I went with that results in two DB queries instead of one which is fine for my situation.
var entries = context.CompletedCourses
.Include(x => x.Agent)
.Include(x => x.Course);
var courses = entries.ToList();
var courseIds = entries.Select(x => x.CompletedCourseId);
var licenses =
(from course in entries
join license in context.Licenses
on new { course.AgentId, course.Course.StateId }
equals new { AgentId = license.UserId, license.StateId }
where courseIds.Contains(course.CompletedCourseId)
select license);
foreach (var course in courses)
{
var license = agentLicenses.FirstOrDefault(x => x.UserId == course.AgentId &&
x.StateId == course.Course.StateId);
if (license != null)
{
course.LicenseNumber = license.LicenseNumber;
}
}
return courses;
select TeamName, [Description], COUNT(u.UserId)
from Team t
left outer join [User] u on u.TeamId=t.TeamId
group by TeamName, Description, UserId
and here i have so far but cant able to do that.please help
var countUser = (from t in db.Teams
join u in db.Users on u.TeamId equals t.TeamId
group TeamName, Description, UserId by select
new
{
u.UserId
}).Count();
This should do it:
Teams.Join(Users.DefaultIfEmpty().
t => t.TeamId,
u => u.TeamId,
(t, u) => new { t.TeamName, t.Description, UserId = u == null ? null:(int?)u.UserId })
.GroupBy(x => x)
.Select(g => new { g.Key.TeamName, g.Key.Description, Count = g.Count() });
RePierre I'm going to steal part of your answer (+1) because I think I understand what OP is talking about, though the question text does not convey it.
You could do something like this:
// Model class for View
public class UsersPerTeamCount
{
public string TeamName { get; set; }
public string Description { get; set; }
public int Count { get; set; }
}
// ...
public ActionResult PlayersPerTeam()
{
var model = from t in db.Teams
join u in db.Users on t.TeamId equals u.TeamId into joinedRecords
select new UsersPerTeamCount()
{
Name = t.TeamName,
Description = t.Description,
PlayerCount = joinedRecords.Count()
};
return View(model);
}
As far as in OPs comment "please try to write as like..." that's just a difference in syntax, it doesn't really matter which way you write it - either fluent vs query syntax (at least i think it's called query syntax)