Develop Reference

How to filter child table by dictionary

I have a question for you all!!!,
Assume that I have 2 tables, one of them is a parent, and the other is a child. I want to filter the parent table by sending data from the child object. The child table contains 4 columns like
{Id, RuleId, RuleKey, RuleValue}
I send a dictionary as a request that includes value for 2 columns ( RuleKey, RuleValue) of the child object. I want to return the parent object that includes RuleKey and RuleValue I sent.
// Key is the column name of the child table and value is the corresponding value
var dictionary = new Dictionary<string,string>()
{
{"RuleKey", "RuleValue"},
{"RuleKey1", "RuleValue1"}
}
This is what I tried but it obviously won't work and still give the syntax error.
These 2 questions about this issue;
1-) How should I do this (Important one:)))))
2-) Will be there any error If I do this In-Memory, If I remember that correctly, EFCore won't translate the LINQ to SQL script because of the memory issues.
var rules = _ruleContext.Rules.AsQueryable();
rules = rules.Where(u => u.Criteria.Where(k => dictionary.Keys.Contains(k.Key) && dictionary.Values.Contains(k.Value)));
public class Rule : Entity<int>
{
public string RuleName { get; set; }
public string Expression { get; set; }
public ICollection<Criteria> Criteria { get; set; }
}
public class Criteria : Entity<int>
{
public int RuleId { get; set; }
public string Name { get; set; }
public string Key { get; set; }
public string Operator { get; set; }
public string Value { get; set; }
public Rule Rule { get; set; }
}

I'm assuming that you want to only match criteria where both Key and Value match a key/value pair in the dictionary. (Your pseudocode would include any rules where either the key or value matched)
What you're trying to do is join between the Criteria table and the key/values in the dictionary. One way to accomplish this would be to create a temporary table containing those pairs, and then use a raw SQL statement to join from Criteria onto that.
Something like this might work:
// To use the temporary table you need to make sure that the connection stays
// open beyond the table creation statement
context.Database.OpenConnection();
try
{
context.Database.ExecuteSqlRaw("CREATE TABLE #RequiredCriteria ([Value] nvarchar(100), [Key] nvarchar(100))");
// If you're expecting LOTs of criteria this could become a bottleneck...
foreach (var criteria in dictionary)
{
context.Database.ExecuteSqlInterpolated($"INSERT INTO #RequiredCriteria ([Value], [Key]) VALUES ({criteria.Key}, {criteria.Value})");
}
// This creates Rule objects from a raw SQL query that does
// the required joining of parameters
var matched = context.Rules.FromSqlRaw(
#"SELECT * FROM Rules
WHERE Id IN (
SELECT DISTINCT RuleId
FROM Criteria C
INNER JOIN #RequiredCriteria RC
ON C.[Key] = RC.[Key] AND C.[Value] = RC.[Value])")
.ToList();
context.Database.ExecuteSqlRaw("DROP TABLE #RequiredCriteria");
}
finally
{
context.Database.CloseConnection();
}
Edit (now without distinct):
I noticed in your comment that a rule must meet all the given criteria - this version of the query will make sure that any matching rule will have the same number of distinct criteria that were requested. That way if the same criteria key and value is duplicated in the request it won't matter:
// To use the temporary table you need to make sure that the connection stays
// open beyond the table creation statement
context.Database.OpenConnection();
try
{
context.Database.ExecuteSqlRaw("CREATE TABLE #RequiredCriteria ([Value] nvarchar(100), [Key] nvarchar(100))");
// If you're expecting LOTs of criteria this could become a bottleneck...
foreach (var item in dictionary)
{
context.Database.ExecuteSqlInterpolated($"INSERT INTO #RequiredCriteria ([Value], [Key]) VALUES ({item.Key}, {item.Value})");
}
var matched = context.Rules.FromSqlInterpolated(
#$"SELECT * FROM Rules
WHERE Id IN (
SELECT RuleId
FROM Criteria C
INNER JOIN #RequiredCriteria RC
ON C.[Key] = RC.[Key] AND C.[Value] = RC.[Value]
GROUP BY RuleId
HAVING COUNT(RuleId) = {dictionary.Count}
)")
.ToList();
context.Database.ExecuteSqlRaw("DROP TABLE #RequiredCriteria");
}
finally
{
context.Database.CloseConnection();
}
Edit 2:
If the Criteria table is going to be quite large, I'd definitely recommend you have an index on the Key and Value columns, otherwise you'll be doing a clustered index scan across the entire table for every query.

You can use function FilterByItems (don't want to repeat myself). Then query can be written in the following way:
var dictionary = new Dictionary<string,string>()
{
{"RuleKey", "RuleValue"},
{"RuleKey1", "RuleValue1"}
}
var matched = _ruleContext.Criterias
.FilterByItems(dictionary, (c, kv) => c.Key == kv.Key && c.Value == kv.Value, true)
.GroupBy(c => c.RuleId)
.Where(g => g.Count() == dictionary.Count)
.Select(g => new
{
RuleId = g.Key,
});
var rules =
from r in _ruleContext.Rules
join m in matched on r.Id equals m.RuleId
select r;
var result = rules.ToArray();

Simple outer apply not working in Entity Framework 6

I have a domain class like below :
public class Employee
{
public int EmployeeId { get; set; }
public int DeptId { get; set; }
}
public class Transaction
{
public int TRID { get; set; }
public int EmployeeId { get; set; }
public string Status { get; set; }
}
Now I want to get all employees from the EmployeeTable for DeptId = 100. I want to calculate Pending status for those employees whose transactions are pending.
So if employee records are found in Transactions table then just want to return a column saying whether employee has any pending transactions or not)
Query :
var t = (from e in _employeeRepository.GetAll() //returns IQueryable<Employee>
where e.DeptId == 100
from t in _transactionRepository.GetAll().Where(t => t.EmployeeId == e.EmployeeId)
select new
{
IsPendingTransaction = (t != null && t.Status != "Done") ? true : false,
}).ToList();
Error : LINQ to Entities does not recognize the method
'System.Linq.IQueryable`1[Transaction] GetAll()' method, and this
method cannot be translated into a store expression."}
Sql Query :
SELECT e.*
(CASE WHEN (t.EmployeeId is not null and t.Status <> 'Done')
THEN CAST(1 AS BIT)
ELSE CAST(0 AS BIT)
End) as IsPendingTransaction
FROM Employee e OUTER APPLY
(SELECT t.*
FROM Transactions t
WHERE e.EmployeeId = t.EmployeeId
) t
WHERE e.DeptId = 100;

The issue is that when you work within IQueryable, every statement inside that Linq expression must be understood by EF to be able to be translated to SQL.
Your first repository call returns an IQueryable<Employee> which you are trying to extend by telling it to join on some code called "_transactionRepository.GetAll()" EF doesn't know what this is, it doesn't correlate to mapped DbSets or properties on entities...
If your Transaction entity has a navigation property back to Employee (which it should) you should be able to accomplish what you want using just the TransactionRepository with something like:
var t = _transactionRepository.GetAll()
.Where(t => t.Employee.DeptId == 100)
.Select(t => new
{
IsPendingTransaction = (t != null && t.Status != "Done") ? true : false
}).ToList();
Using IQueryable<TEntity> in a repository pattern can be quite powerful, however I don't recommend adopting a Generic repository pattern as it just serves to fragment your thinking when working with entities and their relationships with one another, allowing EF to manage the resulting SQL without you resorting to pre-emptively trying to do the joining yourself, often causing conflicts with what EF is capable of working out itself.
Edit: Ok, from your description to get a list of employees with a flag if they have a pending transaction: That would be back at the Employee level with a query something like:
var employees = _employeeRepository.GetAll()
.Where(e => e.DeptId == 100)
.Select(e =>
{
Employee = e,
HasPendingTransaction = e.Transactions.Any(t => t.Status != "Done")
}).ToList();
Or projected to a ViewModel to embed the HasPendingTransaction alongside the Employee details:
var employees = _employeeRepository.GetAll()
.Where(e => e.DeptId == 100)
.Select(e => new EmployeeDetailsViewModel
{
EmployeeId = e.EmployeeId,
Name = e.Name,
// include other relevent details needed for the view...
HasPendingTransaction = e.Transactions.Any(t => t.Status != "Done")
}).ToList();
The advantage of projection is you can build more efficient / faster queries that reduce the amount of data sent over the wire and avoid issues like lazy load trips if you try to serialize entities to the view.

Fix Transaction class
public class Transaction
{
public int TRID { get; set; }
public string Status { get; set; }
public int EmployeeId { get; set; }
public virtual Employee Employee { get; set; }
}
It is not the best idea to have a separate repository for each entity since query usually consists from several entities. It is better to make a join using dbcontext then several repository queries as you trying to do. Don't try to create a base generic repository also. Sooner or later you will see that is is not very helpfull. So add in one of your repositories (probably EmployeeRepository) query like this
var employees= dbcontext.Transactions
.Where(t=> t.Employee.DeptId == 100 && t.EmployeeId==employeeId)
.Select (t=> new {
EmployeeName= t.Employee.Name,
IsPendingTransaction = (t.Status != null && t.Status != "Done") ? true : false​}).ToList()

Need to convert Left join SQL to linq query - help appreciated

Here is my code the issue I have is the less than comparison in the On clause ... Since Linq doesn't allow this .... Migrating down into the where clause wont work as I am comparing one of the fields to null.
Here is the sql query (THE a.UserID= is hardcoded for now)
SELECT A.Policy, A.Comments, A.EventDTTM, A.Status, A.Reason, A.FollowUp
FROM PP_PolicyActivity A
LEFT JOIN PP_PolicyActivity B
ON(A.Policy = B.Policy AND A.EventDTTM < B.EventDTTM)
WHERE A.UserID = 'Ixxxxxx'
AND B.EventDTTM IS NULL AND a.status = 'open - Pending'
order by A.EventDTTM DESC
I need the result set from the above query as an IEnumerable list to populate a view
I'm tasked with rebuilding an old VB ASP NET that has a set of standing production databases behind it ... i don't have the option of changing the db design. I connecting to the server and database and this query was going against a table on that database.. the model also reflects the layout of the actual table.
The problem is with A.EventDTTM < B.EventDTTM - I can't move this to the where clause as I also have to deal with B.EventDTTM IS NULL in the where clause.
I need to retool the query someway so that it is 'linq' friendly
public class PolicyActivityModel
{
public string Policy { get; set; }
public int PolicyID { get; set; }
public string Status { get; set; }
public string Reason { get; set; }
public string Comments { get; set; }
public DateTime EventDTTM { get; set; }
public string UserID { get; set; }
public DateTime FollowUp { get; set; }
}
Company policy prohibits me from showing the connection string.
I am extremely new to Linq, Any help greatly appreciated
thank you

You can use the navigation property after you get the policy from the database.
var policy = DbContext.First(x => x.Id == 1000);
var otherPolicies = policy.ConnectedPolicies.Where(p => ...);

It's weird being a self-join but this is the most direct translation to Linq:
var query = from leftPP in PP_PolicyActivity
join rightPP in PP_PolicyActivity
on new { Policy = leftPP.Policy, EventDTTM = leftPP.EventDTTM }
equals new { Policy = rightPP.Policy, EventDTTM = rightPP.EventDTTM }
into pp from joinedRecords.DefaultIfEmpty()
where leftPP.UserId == 1
&& leftPP.EventDTTM < rightPP.DTTM)
&& rightPP.EventDTTM == null
&& leftPP.status = "open - Pending"
select new
{
leftPP,
rightPP
}
I free typed this, without models or Intellisense, thus there might be some smaller errors.
You could add the order by in that clause, but it's also still an IQUeryable, so I'd leave it.
And then, to get a List of models:
var results = query.OrderByDescending(x => x.EventDTTM).ToList();
The actual join is lines 2,3,4 and 5. It's verbose and "backwards" from SQL, and most importantly uses anonymous types. Accessing indidual properties will look something like:
results[0].leftPP.PolicyId

EF Core - translating to SQL simple select

I have very simple models
.NET Core 2.1 / EF Core 2.1 / MSSQL
public class ImageZ
{
[Key]
public Guid Id { get; set; }
public string Base64 { get; set; }
public string Title { get; set; }
}
public class Gallery
{
[Key]
public Guid Id { get; set; }
public ImageZ MainImage { get; set; }
public List<ImageZ> Images { get; set; } = new List<ImageZ>();
}
and I'm using this LINQ to load it from db
return _context
.Gallery
.Include(x => x.Images)
.Include(x => x.MainImage)
.OrderBy(x => Guid.NewGuid())
.FirstOrDefault();
But it sends two queries to db
SELECT TOP(1) [x].[Id], [x].[MainImageId], [x.MainImage].[Id], [x.MainImage].[Base64], [x.MainImage].[GalleryId], [x.MainImage].[Title]
FROM [Gallery] AS [x]
LEFT JOIN [ImageZ] AS [x.MainImage] ON [x].[MainImageId] = [x.MainImage].[Id]
ORDER BY NEWID(), [x].[Id]
SELECT [x.Images].[Id], [x.Images].[Base64], [x.Images].[GalleryId], [x.Images].[Title]
FROM [ImageZ] AS [x.Images]
INNER JOIN (
SELECT DISTINCT [t].*
FROM (
SELECT TOP(1) [x0].[Id], NEWID() AS [c]
FROM [Gallery] AS [x0]
LEFT JOIN [ImageZ] AS [x.MainImage0] ON [x0].[MainImageId] = [x.MainImage0].[Id]
ORDER BY [c], [x0].[Id]
) AS [t]
) AS [t0] ON [x.Images].[GalleryId] = [t0].[Id]
ORDER BY [t0].[c], [t0].[Id]
Is it correct behaviour? shouldn't it be done just with one?

Do you really need to select * from MainImage and Images? The blanket include could have an impact on your index selection. With 2.2, you can now use a projection with .ToList() to only select the columns you need. It will still use separate queries for each child collection, but will be limited to the columns you project into.
Alternatively, since you are only selecting one row (at random due to the order by on New Guid), you might be able to issue the separate requests explicitly and not need the order by in the secondary query (over Images). Indeed since each query will be using a different newid(), I suspect your images results in this case won't be properly aligned and you may need to do it explicitly.

ServiceStack LoadReferences when using SQL Query

Is it possible to Load References when instead of using the code below:
SqlExpression<Customer> q = db.From<Customer>();
q.Join<Customer,CustomerAddress>((cust,address) => cust.Id == address.CustomerId);
List<Customer> dbCustomers = db.LoadSelect(q);
Using this:
public class KpiTotal : IKpiTotal
{
public DateTime Date { get; set; }
public int TeamId { get; set; }
public Team Team { get; set; }
public int AccountId { get; set; }
public Account Account { get; set; }
public double Total { get; set; }
}
var result = dbCon.SelectFmt<KpiTotal>(#"select convert(date, t.TransactionDate) [Date], tm.TeamId,a.AccountNumber, count(distinct(t.RequisitionNumber)) Total
from task.tblTransactions t
inner join task.tblRequisitions r on r.RequisitionNumber = t.RequisitionNumber
inner join task.tblAccounts a on a.AccountNumber = r.AccountNumber
inner join Team tm on tm.DivisionId = a.DivisionId
where t.TransactionTypeNumber = 201 and a.IsActive = 1
and t.TransactionDate between {0} and {1}
group by convert(date, t.TransactionDate), tm.TeamName, a.AccountName
order by 1,2 desc", dateRange.Start, dateRange.End);
Because my result object (KpiTotal) has references to two child tables, and I would like to automatic load the references, instead of getting it with a foreach block.

I'm assuming you want to load in Team and Account from the above query. The LoadSelect method sniffs the POCO model and generates a query that pulls back all related DB records based on the foreign key relationships to the core object you're querying. It generates a query similar to this for each referenced / joined POCO (very pseudo-coded):
SELECT * FROM Team /* Related POCO */
WHERE Team.Id IN (SELECT TeamId FROM [original query with WHERE clase])
Basically, it does a single query to bring back all Teamss or Accounts.
With ServiceStack.OrmLite v4.0.40, there is now a new Merge extension method that will stitch together object references based in a more manual process.
In your case, you can query your KpiTotal results, then run just two separate queries to fetch back Team and Account lists, then merge them in. Basically:
var result = dbCon.SelectFmt<KpiTotal>(/* gnarly SQL */);
var teams = dbCon.Select(/* get all relevant teams */);
var accounts = dbCon.Select(/* get all relevant accounts */);
result.Merge(teams);
result.Merge(accounts);
Debug.WriteLine(result.Dump()); // Output to console / debug window, whatever