Develop Reference

How to add select in my Include with a theninclude

I'm trying to get info from 3 tables, Clients, invoices, and transactions. Every client can have many invoices, and every invoice can have many transactions.
There are several hundreds of thousands of the invoices and transactions with quite a few columns on each table. I only need a few of the columns and would take too long to grab all columns on all the tables.
Here's what I'm attempting to do, but it does not work.
var Clients = cps.Clients
.Include(q => q.Invoices.Select(x=> new TInvoices
{
Invno = x.Invno,
// more columns...
})
.ThenInclude(q => q.Transactions).ToList();
I'd also like to do a select on the transactions as well.
Any thoughts on how to do this?
I was going to try some linq joins and such, but I have limited knowledge and understanding of that area. Open to suggestions or ideas!
Thanks!

Use Select instead of Include. Include always loads whole record and there is no way to do something else.
var Clients = cps.Clients
.Select(c => new // or other concrete type
{
c.id,
// other properties
Invoices = c.Invoices.Select(i => new TInvoices
{
Invno = i.Invno,
// more columns...
Transations = i.Transactions.ToList()
})
.ToList()
});

How to sort/filter a list in the same way you sort a database result

I am currently coding with .net core 5 preview 3 and I am having an issue with filtering a list of best matched customers.
Given these two different code samples how come they produce different results?
How can I fix the second sample to return the same results as the first sample?
Sample One (this works)
//This properly gives the top 10 best matches from the database
using (var context = new CustomerContext(_contextOptions))
{
customers = await context.vCustomer.Where(c => c.Account_Number__c.Contains(searchTerm))
.Select(c => new
{
vCustomer = c,
MatchEvaluator = searchTerm.Contains(c.Account_Number__c)
})
.OrderByDescending(c => c.MatchEvaluator)
.Select(c => new CustomerModel
{
CustomerId = c.vCustomer.Account_Number__c,
CustomerName = c.vCustomer.Name
})
.Take(10)
.ToListAsync();
}
Customer Id Results from sample one (these are the best results)
247
2470
247105
247109
247110
247111
247112
247113
247116
247117
Sample Two (This doesn't work the same even though its the same code)
//this take all customers from database and puts them in a list so they can be cached and sorted on later.
List<CustomerModel> customers = new List<CustomerModel>();
using (var context = new CustomerContext(_contextOptions))
{
customers = await context.vCustomer
.Select(c => new CustomerModel
{
CustomerId = c.Account_Number__c,
CustomerName = c.Name
})
.ToListAsync();
}
//This does not properly gives the top 10 best matches from the list that was generated from the database
List<CustomerModel> bestMatchedCustomers = await Task.FromResult(
customers.Where(c => c.CustomerId.Contains(searchTerm))
.Select(c => new
{
Customer = c,
MatchEvaluator = searchTerm.Contains(c.CustomerId)
})
.OrderByDescending(c => c.MatchEvaluator)
.Select(c => new CustomerModel
{
CustomerId = c.Customer.CustomerId,
CustomerName = c.Customer.CustomerName
})
.Take(10)
.ToList()
);
Customer Id Results from sample two
247
1065247
247610
32470
324795
624749
762471
271247
247840
724732

You asked "why are they different" and for this you need to appreciate that databases have a optimizer that looks at the query being run and changes its data access strategy according to various things like how many records are being selected, whether indexes apply, what sorting is requested etc
One of your queries selects all the database table into the client side list and then uses the list to do the filter and sort, the other uses the database to do the filter and the sort. To a database these will be very different things; hitting a table you likely get the rows out in the order they're stored on disk, which could be random. Using a filter you might see the database using some indexing strategy where it includes/discounts a large number of rows based on an index, or it might even use the index to retrieve the requested data. How it then sorts the ties, if it does, might be completely different to how the client side list sorts ties (does nothing with them actually). Either way, the important point is the database is planning and executing your two different queries differently. It sees different queries because your second version runs the query without a where or order by
When you couple this up with your sort operation being on a column that is incredibly cardinality (how unique the values in the column are) i.e. your lead result, the one where the record equals the search term, is 1 and EVERYTHING else is 0. This means that one record bubbles to the top then the rest of the records are free to be sorted however the system doing the sorting likes, and then you take a subset of them
..hence why one looks like X and the other like Y
If you didn't take the subset the two datasets would be in different orders but everything in set 1 would be in set 2 somewhere... it's just that one set is like 1 3 5 7 2 4 6, the other is like 1 7 6 5 4 3 2, you're taking the first three results and asking "why is 1 3 5 different to 1 7 6"
In terms of your code, I think I would have just done something simple that also sorts in a stable fashion (rows in same order because there is no ambiguity/ties) like:
await context.vCustomer
.Where(c => c.Account_Number__c.Contains(searchTerm))
.OrderBy(c => c.Account_Number__c.Length)
.ThenBy(c => c.Account_Number__c) //stable, if unique
.Take(10)
.Select(c => new CustomerModel
{
CustomerId = c.vCustomer.Account_Number__c,
CustomerName = c.vCustomer.Name
}
)
.ToListAsync();
If you sort the results by their length in chars then 247 is better match than 2470, which is better than 24711 or 12471 etc
"Contains" can be quite performance penalising; perhaps consider StartsWith; theoretically at least, an index could still be used for that
ps: calling your var a MatchEvaluator makes things really confusing for people who know regex well btw

You're ordering by the MatchEvaluator value which is either 1 or 0.
If I understood correctly what you want to do is first order by the MatchEvaluator and then by the CustomerId:
List<CustomerModel> bestMatchedCustomers =
await Task.FromResult(
customers.Where(c => c.CustomerId.Contains(searchTerm))
.OrderBy(c => c.CustomerId.IndexOf(searchTerm))
.ThenBy(c => c.CustomerId)
.Select(c => new CustomerModel
{
CustomerId = c.Customer.CustomerId,
CustomerName = c.Customer.CustomerId
})
.Take(10)
.ToList()
);

LINQ: Is there a way to combine these queries into one?

I have a database that contains 3 tables:
Phones
PhoneListings
PhoneConditions
PhoneListings has a FK from the Phones table(PhoneID), and a FK from the Phone Conditions table(conditionID)
I am working on a function that adds a Phone Listing to the user's cart, and returns all of the necessary information for the user. The phone make and model are contained in the PHONES table, and the details about the Condition are contained in the PhoneConditions table.
Currently I am using 3 queries to obtain all the neccesary information. Is there a way to combine all of this into one query?
public ActionResult phoneAdd(int listingID, int qty)
{
ShoppingBasket myBasket = new ShoppingBasket();
string BasketID = myBasket.GetBasketID(this.HttpContext);
var PhoneListingQuery = (from x in myDB.phoneListings
where x.phonelistingID == listingID
select x).Single();
var PhoneCondition = myDB.phoneConditions
.Where(x => x.conditionID == PhoneListingQuery.phonelistingID).Single();
var PhoneDataQuery = (from ph in myDB.Phones
where ph.PhoneID == PhoneListingQuery.phonePageID
select ph).SingleOrDefault();
}

You could project the result into an anonymous class, or a Tuple, or even a custom shaped entity in a single line, however the overall database performance might not be any better:
var phoneObjects = myDB.phoneListings
.Where(pl => pl.phonelistingID == listingID)
.Select(pl => new
{
PhoneListingQuery = pl,
PhoneCondition = myDB.phoneConditions
.Single(pc => pc.conditionID == pl.phonelistingID),
PhoneDataQuery = myDB.Phones
.SingleOrDefault(ph => ph.PhoneID == pl.phonePageID)
})
.Single();
// Access phoneObjects.PhoneListingQuery / PhoneCondition / PhoneDataQuery as needed
There are also slightly more compact overloads of the LINQ Single and SingleOrDefault extensions which take a predicate as a parameter, which will help reduce the code slightly.
Edit
As an alternative to multiple retrievals from the ORM DbContext, or doing explicit manual Joins, if you set up navigation relationships between entities in your model via the navigable join keys (usually the Foreign Keys in the underlying tables), you can specify the depth of fetch with an eager load, using Include:
var phoneListingWithAssociations = myDB.phoneListings
.Include(pl => pl.PhoneConditions)
.Include(pl => pl.Phones)
.Single(pl => pl.phonelistingID == listingID);
Which will return the entity graph in phoneListingWithAssociations
(Assuming foreign keys PhoneListing.phonePageID => Phones.phoneId and
PhoneCondition.conditionID => PhoneListing.phonelistingID)

You should be able to pull it all in one query with join, I think.
But as pointed out you might not achieve alot of speed from this, as you are just picking the first match and then moving on, not really doing any inner comparisons.
If you know there exist atleast one data point in each table then you might aswell pull all at the same time. if not then waiting with the "sub queries" is nice as done by StuartLC.
var Phone = (from a in myDB.phoneListings
join b in myDB.phoneConditions on a.phonelistingID equals b.conditionID
join c in ph in myDB.Phones on a.phonePageID equals c.PhoneID
where
a.phonelistingID == listingID
select new {
Listing = a,
Condition = b,
Data = c
}).FirstOrDefault();
FirstOrDefault because single throws error if there exists more than one element.

LINQ to SQL with a large database table increase performance

I am doing a search on a database table using a wildcard(The contains extension) Here is the code
// This gets a list of the primary key IDs in a table that has 5000+ plus records in it
List<int> ids = context.Where(m => m.name.ToLower().Contains(searchTerm.ToLower())).Select(m => m.Id).ToList();
// Loop through all the ids and get the ones that match in a different table (So basically the FKs..)
foreach (int idin nameId)
{
total.AddRange(context2.Where(x => x.NameID == id).Select(m => m.Id).ToList());
}
In there anything I could change in the LINQ that would result in getting the IDs faster?
Thanks

I have not tested it, but you could do something along these lines:
var total =
from obj2 in context2
join obj1 in context1 on obj2.NameID equals obj1.Id
where obj1.name.ToLower().Contains(searchTerm.ToLower())
select obj2.Id
It joins the two tables, performing a cartesian product first and then limiting it to the pairs where the NameIds match (see this tutorial on the join clause). The where line does the actual filtering.
It should be faster because the whole matching is done in the database, and only the correct ids are returned.
If you had a Name property in the context2 item class that holds a reference to the context1 item, you could write it more readable:
var total = context2
.Where(x => x.Name.name.ToLower().Contains(searchTerm.toLower()))
.Select(x => x.ID);
In this case, Linq to SQL would do the join automatically for you.

In terms of performance you can see tests that show if you search 1 string in 1 000 000 entries its around 100 ms.
Here is the link with tests and implementation.
for (int y = 0; y < sf.Length; y++)
{
c[y] += ss.Where(o => o.Contains(sf[y])).Count();
}

Linking Multiple Tables in LINQ to SQL

I would like to get the list of albums (Distinct) which was sung by the artistId=1
I am very new to LINQ to SQL and do not know how to join multiple tables. Please see the database diagram below:
alt text http://a.imageshack.us/img155/8572/13690801.jpg
SingBy is the middle table between Track and Artist.
How could I achieve this?

var albums = from singer in artist
from sb in singby
from t in track
from a in album
where singer.artistId == 1 &&
sb.artistId == 1 &&
sb.trackId == t.trackId &&
a.albumId == track.albumId
select a;
I'm sure there must be a better way. You should look into creating Navigation Properties on your entities. Navigation Properties are like foreign keys.
Edit - corrected to get albums, not artists.

Now, I wrote the codes like the following and it works.
var albums = (from a in db.artists
where a.artistId == 1
join sb in db.singbies on a equals sb.artist
join t in db.tracks on sb.track equals t
join al in db.albums on t.album equals al
select al).Distinct();
return albums.ToList() as List<album>;
I tested the Chad's version and it works too. I would like to know which way is better and good for query optimization? Thanks all.

If you have all the foreign key relationship defined, you should be able to issue call like below:
dc.GetTable<Album>().Where(a => a.Track.Singby.ArtistId == 1).ToList();
This is relying on Linq to perform lazy load for Track and Singby automatically when required. Obviously this is not optimal to use when you have a large set of data in the db and performance is critical. You can chain the query with GroupBy or Distinct operation to return only the distinct set such as
dc.GetTable<Album>().Where(a => a.Track.Singby.ArtistId == 1).Distinct().ToList();

I would like to get the list of albums
(Distinct) which was sung by the
artistId=1
DBDataContext = new DBDataContext();
album[] = db.artists.Where(a => a.artistId == 1) /* Your artist */
.SelectMany(a => a.singbies) /* Check if `singby` converted to `singbies` */
.Select(sb => sb.track) /* The tracks */
.Select(t => t.album) /* The albums */
.GroupBy(al => al.albumId) /* Group by id */ /* "Distinct" for objects */
.Select(alG => alG.First()) /* Select first of each group */
.ToArray();

IEnumerable<Album> query =
from album in myDC.Albums
let artists =
from track in album.Tracks
from singBy in track.SingBys
select singBy.Artist
where artists.Any(artist => artist.ArtistId == 1)
select album;

List<int> Ids = dc.Albums.Where(a => a.Track.Singby.ArtistId == 1).Select(a=> a.albumId).Distinct().ToList();
List<Album> distinctAlbums = dc.Albums.Where(a => distinctAlbumIds.Contains(a.albumId)).ToList();
Hey TTCG, above is the simplest way to do it. This is because doing a Distinct on a List of objects won't do it based on the albumId.
Either you do it in two steps as above, or, you write your own Album Comparer which specifies uniqueness based on AlbumId and pass it to the Distinct call on a List.
NOTE:
The above will only work if you've defined the constraints in your DBML, but better still in your DB.
For best practices, always define your relationships IN THE DATABASE when using Linq to SQL, as Linq to SQL is not like EF, or NHibernate, in that is does not "abstract" your db, it simply reflects it. It's a tool for Data Driven Design, not Domain Driven, so define the relationships in the db.