I have 3 kinds of objects: Agency, BusinessUnit and Client (each with their own respective table)
In terms of hierarchy, Agencies own BusinessUnits, and BusinessUnits own Clients.
I have 3 C# POCO Objects to represent them (I usually select new {} into them, rather than use the LINQ generated classes):
public class Agency
{
public IEnumerable<BusinessUnit> BusinessUnits { get; set; }
}
public class BusinessUnit
{
public IEnumerable<Client> Clients { get; set; }
}
public class Client
{
public int NumberOfAccounts { get; set; }
public Decimal AmountOfPlacement { get; set; }
public Decimal AvgBalance { get; set; }
public Double NeuPlacementScore { get; set; }
}
You can see that Agencies contain a list of BusinessUnits, and BusinessUnits contain a list of Clients.
I also have a mapping table called BAC_Map in the database which says which owns which, and it looks something like this:
How can I construct a query, so I can query for and return a list of Agencies? Meaning that, I want each Agency to have its list of BusinessUnit objects set, and I want the list of BusinessObjects to have its list of Clients set.
I can do basic LINQ queries, but this is a little over my head concerning the Map table and the multiple? queries.
How could I construct a method like GetAllAgencies() which would query, for not only all agencies, but populate its BusinessUnits that Agency owns, and the Clients those BusinessUnits own?
Edit: Any tips or info is appreciated. Do I need to do joins? Does this need to be multiple queries to return an Agency list, with its submembers populated?
If you drop all four tables (Agency, BusinessUnit, Client, Map) on the linq to sql designer, and draw relationships from Map to the other three, there will be some useful properties on Map.
//construct a query to fetch the row/column shaped results.
var query =
from m in db.map
//where m.... ?
let a = m.Agency
let b = m.BusinessUnit
let c = m.Client
// where something about a or b or c ?
select new {
AgencyID = a.AgencyID,
AgencyName = a.Name,
BusinessUnitID = b.BusinessUnitID,
ClientID = c.ClientID,
NumberOfAccounts = c.NumberOfAccounts,
Score = c.Score
};
//hit the database
var rawRecords = query.ToList();
//shape the results further into a hierarchy.
List<Agency> results = rawRecords
.GroupBy(x => x.AgencyID)
.Select(g => new Agency()
{
Name = g.First().AgencyName,
BusinessUnits = g
.GroupBy(y => y.BusinessUnitID)
.Select(g2 => new BusinessUnit()
{
Clients = g2
.Select(z => new Client()
{
NumberOfAccounts = z.NumberOfAccounts,
Score = z.Score
})
})
})
.ToList();
If approriate filters are supplied (see the commented out where clauses), then only the needed portions of the tables will be pulled into memory. This is standard SQL joining at work here.
I created your tables in a SQL Server database, and tried to recreate your scenario in LinqPad. I ended up with the following LINQ statements, which basically result in the same structure of your POCO classes:
var map = from bac in BAC_Maps
join a in Agencies on bac.Agency_ID equals a.Agency_ID
join b in BusinessUnits on bac.Business_Unit_ID equals b.Business_Unit_ID
join c in Clients on bac.Client_ID equals c.Client_ID
select new
{
AgencyID = a.Agency_ID,
BusinessUnitID = b.Business_Unit_ID,
Client = c
};
var results = from m in map.ToList()
group m by m.AgencyID into g
select new
{
BusinessUnits = from m2 in g
group m2 by m2.BusinessUnitID into g2
select new
{
Clients = from m3 in g2
select m3.Client
}
};
results.Dump();
Note that I called map.ToList() in the second query. This actually resulted in a single, efficient query. My initial attempt did not include .ToList(), and resulted in nine separate queries to produce the same results. The query generated by the .ToList() version is as follows:
SELECT [t1].[Agency_ID] AS [AgencyID], [t2].[Business_Unit_ID] AS [BusinessUnitID], [t3].[Client_ID], [t3].[NumberOfAccounts], [t3].[AmountOfPlacement], [t3].[AvgBalance], [t3].[NeuPlacementScore]
FROM [BAC_Map] AS [t0]
INNER JOIN [Agencies] AS [t1] ON [t0].[Agency_ID] = [t1].[Agency_ID]
INNER JOIN [BusinessUnits] AS [t2] ON [t0].[Business_Unit_ID] = [t2].[Business_Unit_ID]
INNER JOIN [Clients] AS [t3] ON [t0].[Client_ID] = [t3].[Client_ID]
Here is a screenshot of the results:
alt text http://img411.imageshack.us/img411/5003/agencybusinessunitclien.png
If you are doing this with direct LINQ to SQL, there is no way to do this without some kind of recursion, whether you do it yourself or you hide it behind an extension method. Recursive SQL is very bad (many round trips, many single queries).
There are two options here. One is to pull the entire table(s) with the hierarchy into memory and use LINQ to Objects on it. Leave the "details" tables in SQL. If you have less than several thousand entities, this is probably the most efficient way to go. You can keep a single copy of the table(s) in cache and refresh them when necessary. When you need to fetch more detailed data from the DB for a single record, you can reattach that entity from your cached hierarchy to a new DataContext and fetch it.
The other option is to use a more complex relationship model in your database. Storing parent only by nature demands recursion, but you can use the adjacency list model to construct a single query which can span many levels of inheritance. This will mean your LINQ to SQL queries become less intuitive (querying against Entity.Right and Entity.Left isn't quite as pretty as Parent or Children...) but you can do in one query what might take hundreds or thousands in the literal recursive approach.
Related
I have a linq query like this:
from a in context.table_A
join b in
(from temp in context.table_B
where idlist.Contains(temp.id)
select temp)
on a.seq_id equals b.seq_id into c
where
idlist.Contains(a.id)
select new MyObject
{
...
}).ToList();
idlist is List
The problem I have is that the idlist has too many values (hundreds of thousands to several million records). It works fine with few records but when there are too many records, the contains function is error.
Error log is
The query plan cannot be created due to lack of internal resources of
the query processor. This is a rare event and only occurs for very
complex queries or queries that reference a very large number of
tables or partitions. Make the query easy.
I want to improve the performance of this section. Any ideas?
I would suggest to install extension linq2db.EntityFrameworkCore and use temporary tables with fast BulkCopy
public class IdItem
{
public int Id { get; set; }
}
...
var items = idList.Select(id => new IdItem { Id = id });
using var db = context.CreateLinqToDBConnection();
using var temp = db.CreateTempTable("#IdList", items);
var query =
from a in context.table_A
join id1 in temp on a.Id equals id1.Id
join b in context.table_B on a.seq_id equals b.seq_id
join id2 in temp on b.Id equals id2.Id
select new MyObject
{
...
};
// switch to alternative translator
var query = query.ToLinqToDB();
var result = query.ToList();
I have a table which contains columns among others like Id as Primary Key and LastMeterReadingId (Foreign Key which references to same table) - something like Parent Meter Reading.
I would like to get all rows which are not already used like Parent. I would like to avoid situation when meter reading is parent for more than one meter reading.
I know how to join to same table, but I have no idea how to choose only those records which aren't parent already. That's how query looks like without condition statement.
return (from m in uow.MeterReadingReadWriteRepository.Query()
join parent in uow.MeterReadingReadWriteRepository.Query() on m.Id equals parent.LastMeterReadingId
select new MeterReadingDto()
{
(...)
}).ToList();
Do you have any idea how to achieve it in efficient way?
Regards.
I would like to get all rows which are not already used like Parent
In other words, you want all rows that have no children. Note that the variable name parent in your query is misleading - when you do a join b on a.Id equals b.ParentId, a is the parent and b is the child.
Anyway, there are at least 3 ways to achieve your goal, IMO being equivalent from nowadays database query optimizers point of view (i.e. should be equally efficient):
(1) Using !Any(...) which is equivalent to SQL NOT EXISTS(...):
from m in uow.MeterReadingReadWriteRepository.Query()
where !uow.MeterReadingReadWriteRepository.Query().Any(child => m.Id == child.LastMeterReadingId)
select ...
(2) Using group join:
from m in uow.MeterReadingReadWriteRepository.Query()
join child in uow.MeterReadingReadWriteRepository.Query()
on m.Id equals child.LastMeterReadingId into children
where !children.Any()
select ...
(3) Using left outer antijoin:
from m in uow.MeterReadingReadWriteRepository.Query()
join child in uow.MeterReadingReadWriteRepository.Query()
on m.Id equals child.LastMeterReadingId into children
from child in children.DefaultIfEmpty()
where child == null
select ...
If this is EF (LINQ to Entities), the first two are translated to one and the same SQL NOT EXISTS based query. While the last is translated to the "traditional" SQL LEFT JOIN ... WHERE right.PK IS NULL based query.
You could just add
where !(from child in uow.MeterReadingReadWriteRepository.Query() where child.Id == m.LastMeterReadingId select child).Any()
Not sure how intelligently this would be optimised though. It would also be better to factor out uow.MeterReadingReadWriteRepository.Query().
Do you not have a Child relationship/collection in your Meter Reading entity from the foreign key constraint? - this would make the query much more straightforward.
var readings = uow.MeterReadingReadWriteRepository.Query();
var parents = readings
.Join(readings, child => child.Id, parent => parent.LastMeterReadingId,
(child, parent) => new {parent.Id})
.Distinct()
.ToDictionary(a => a.Id);
var result = (from m in readings
where !parents.Contains(m.Id)
select new
{
Id = m.Id
}).ToList();
Thanks #Ben Jackson
public class MeterReading : EntityBase
{
public long PropertyId { get; set; }
public long? LastMeterReadingId { get; set; }
public long? PaymentId { get; set; }
public Property Property { get; set; }
public MeterReading LastReading { get; set; }
public Payment Payment { get; set; }
}
That's how most value properties looks like. Maybe should I use T-SQL query with JOIN to CTE which mentioned before condition statement? I'll try your solution ASAP.
I have Oracle db with EF 5 on top of it.
Lets say I have tables Company and Orders. I have EF corresponding entities where Company has field
List<Orders> Orders.
I dont want to create an association.
I have a query which is IQuerable and I need to fill the Orders for each company using
order.CompanyId == company.Id
I cant wrap my head around this atm.. Any help will be appreciated.
EDIT: not every company has orders. The orders list could be empty.
I would consider using the .Join() method to include the Orders table. I write this from memory, so please forgive syntax errors.
//I only use this bc I don't know the full schema.
class DTOCompany {
public int ID {get;set;}
public List<Orders> Orders {get;set;}
}
public List<Companies> GetCompaniesOrders() {
using (var db = new Entities()) {
return db.Companies
.AsEnumerable() //may not be needed.
.Join(db.Orders,
c => CompanyId,
o => o.CompanyId,
(c,o) => new { Company = c, Orders = o})
)
.Select(co => new DTOCompany() {
ID = co.Companies.CompanyId,
Orders = co.Orders.ToList()
})
.ToList();
}
}
I'm experimenting with pulling data from multiple datasets using RESTful services. I'm hooking up to the Cloud version of Northwind, and attempting to use Linq to get the equivalent of this:
SELECT TOP 20 p.ProductName, p.ProductID, s.SupplierID, s.CompanyName AS Supplier,
s.ContactName, s.ContactTitle, s.Phone
FROM Products p
JOIN Suppliers s on p.SupplierID = s.SupplierID
ORDER BY ProductName
So, I define a class to hold my data:
public class ProductSuppliers
{
public string ProductName;
public int ProductID;
public string SupplierName;
public string ContactName;
public string ContactPosition;
public string ContactPhone;
}
And hook into the Northwind service:
NorthwindEntities dc = new NorthwindEntities (new
Uri("http://services.odata.org/Northwind/Northwind.svc/"));
After trying to set up a join, not being able to get it to work, and wandering around in the back corridors of MSDN for a while, I find that Linq joins aren't supported by the OData spec. Which seems obvious once you think about it, given the limitations of URI syntax.
Of course, the usual thing to do is stored procs and views on the server side anyway, handling any sort of joins there. However, I wanted to work out some sort of solution for a situation like this one, where you don't have the capability of creating stored procs or views.
My naive solution has all the elegance of medieval battlefield surgery, and it has to scale horribly. I pulled the two tables as two separate List objects, then iterated one, used Find to locate the matching ID in the other, and Added a combined record into my Product. Here's the code:
public List<ProductSuppliers> GetProductSuppliers()
{
var result = new List<ProductSuppliers>();
ProductSuppliers ps;
var prods =
(
from p in dc.Products
orderby p.ProductName
select p
).ToList();
var sups =
(
from s in dc.Suppliers
select s
).ToList();
foreach (var p in prods)
{
int cIndex = sups.IndexOf(sups.Find(x => x.SupplierID == p.SupplierID));
ps = new ProductSuppliers()
{
ProductName = p.ProductName,
ProductID = p.ProductID,
SupplierName = sups[cIndex].CompanyName,
ContactName = sups[cIndex].ContactName,
ContactPosition = sups[cIndex].ContactTitle,
ContactPhone = sups[cIndex].Phone
};
result.Add(ps);
}
return result;
}
There has to be something better than this, doesn't there? Is there something obvious I'm missing?
[Edit] I've looked at the link someone gave me on the Expand method, and that works...sort of. Here's the code change:
var sups =
(
from s in dc.Suppliers.Expand("Products")
select s
).ToList();
This gives me a list of Suppliers with Products for each in a sublist (dc.Suppliers[0].Products[0], etc.). While I could get what I want from there, I'd still have to iterate the entire list to invert the values (wouldn't I?), so it doesn't look like a more scaleable solution. Also, I can't apply Expand to the Products table to include Suppliers (Changing the from clause in prods to from p in dc.Products.Expand("Suppliers") results in a helpful "An Error occurred while processing this request."). So, it doesn't look like I can expand products to include lookup values from Suppliers, since it looks like expanding is expanding parents to include children, not looking up parent values in a list of children. Is there a way to use Expand (or is there some other mechanism besides client-side manipulation of the two tables) to include lookup values from a foreign key table?
The best you can do is described in this SO answer to a similar question. Not what you expected either, since you're required to make multiple roundtrips to the service.
If you don't control the server-side of things (or you don't want to use SPs/views/joins there) you are forced to use one of these mechanisms.
Anyway, at the very least you can improve the products-suppliers matching in your code to this:
var results = from p in prods
join s in sups on s.SupplierId equals p.SupplierId
select new ProductSuppliers()
{
ProductName = p.ProductName,
ProductID = p.ProductID,
SupplierName = s.CompanyName,
ContactName = s.ContactName,
ContactPosition = s.ContactTitle,
ContactPhone = s.Phone
};
You still need to retrieve all records and join in-memory, though.
I am pretty new to Entity Framework and LINQ and I have an entity with more than 10+ other associated entities (one-to-many relationships). Now, I'm planning to make a search page in my application in which users could select which fields (i.e. those 10+ tables) they want to be considered when searching.
Now, I'm trying to write a query to achieve the above goal. Any help how I could sort this out using LINQ method syntax? I mean, to write a multiple join query based on user's choice. (i.e. which of Class1, Class2, ... to join with main Entity to finally have all the related fields in one place). Below is a sample code (Just a hunch, in fact)
if(somefilter#1)
result = db.Companies.Join(db.Channels, p => p.Id, k => k.CId,
(p, k) => new {Company = p, Channels=k});
if(somefilter#2)
result = result.Join(db.BusinnessType, ........);
if(somefilter#3)
result = result.Join(db.Values, .......);
For complex queries it may be easier to use the other LINQ notation. You could join multiple entities like this:
from myEntity in dbContext.MyEntities
join myOtherEntity in dbContext.MyOtherEntities on myEntity.Id equals myOtherEntity.MyEntityId
join oneMoreEntity in dbContext.OneMoreEntities on myEntity.Id equals oneMoreEntity.MyEntityId
select new {
myEntity.Id,
myEntity.Name,
myOtherEntity.OtherProperty,
oneMoreEntity.OneMoreProperty
}
You can join in other entities by adding more join statements.
You can select properties of any entity from your query. The example I provided uses a dynamic class, but you can also define a class (like MyJoinedEntity) into which you can select instead. To do it you would use something like:
...
select new MyJoinedEntity {
Id = myEntity.Id,
Name = myEntity.Name,
OtherProperty = myOtherEntity.OtherProperty,
OneMoreProperty = oneMoreEntity.OneMoreProperty
}
EDIT:
In case when you want to have conditional joins you can define MyJoinedEntity with all the properties you will need if you were to join everything. Then break up the join into multiple methods. Like this:
public IEnumerable<MyJoinedEntity> GetEntities() {
var joinedEntities = from myEntity in dbContext.MyEntities
join myOtherEntity in dbContext.MyOtherEntities on myEntity.Id equals myOtherEntity.MyEntityId
join oneMoreEntity in dbContext.OneMoreEntities on myEntity.Id equals oneMoreEntity.MyEntityId
select new MyJoinedEntity {
Id = myEntity.Id,
Name = myEntity.Name,
OtherProperty = myOtherEntity.OtherProperty,
OneMoreProperty = oneMoreEntity.OneMoreProperty
};
if (condition1) {
joinedEntities = JoinWithRelated(joinedEntities);
}
}
public IEnumerable<MyJoinedEntity> JoinWithRelated(IEnumerable<MyJoinedEntity> joinedEntities) {
return from joinedEntity in joinedEntities
join relatedEntity in dbContext.RelatedEntities on joinedEntity.Id equals relatedEntity.MyEntityId
select new MyJoinedEntity(joinedEntity) {
Comments = relatedEntity.Comments
};
}