Develop Reference

How do I filter duplicates in Entity Framework minimize performance loss?

How can I make the performance better of the code below?
I'm loading the carItems from an external webservice in a list.
carItem is checked whether it exists in EF.
If carItem is new, then it's mapped to carsCol and added to the database. What are some easy ways to improve the performance of this using code?
carItems = carItems.Where(x => x.Name == "Tesla");
// Filter existing cars
List<Car> carsCol = new List<Car>();
foreach (var item in carItems)
{
if (GetById(item.Id) == null)
{
carsCol.Add(item);
}
}
Entities.AddRange(carsCol);

Depending on the situation, you can try to find out which ids already exist in the database by making a single query before the foreach.
var newCarItemIds = carItems.Select(x => x.Id);
var alreadyExistentCarItemIds = Entities.CarItems.Where(x => newCarItemIds.Contains(x.Id)).Select(x=>x.Id);
foreach(var item in carItems)
{
if(!alreadyExistentCarItemIds.Contains(x))
{
carsCol.Add(item);
}
}

You can use Distinct Linq functions: example
You must to implement IEquatable interface

I would propose this. You can do a left outter join with the cars that you get from your API and the existing one. Than you get the new cars that will be added.
var newCars = (from c in carItems
join e in Entities.CarItems on c.Id equals e.Id into g
from x in g.DefaultIfEmpty()
select new { c, IsNew = x == null }).Where(x = x.IsNew).ToList()
With that you do only one access to the database. Also when working with IEnumerable is always good to convert it to either a list or array, that way each time that you iterates through that object you don't run a query in your database.

EF6 IN clause on specific properties

Good morning,
I'm having trouble with a EF query. This is what i am trying to do.
First i am pulling a list of ID's like so (List of IDs are found in the included x.MappingAccts entity):
Entities.DB1.Mapping mapping = null;
using (var db = new Entities.DB1.DB1Conn())
{
mapping = db.Mappings.Where(x => x.Code == code).Include(x => x.MappingAccts).FirstOrDefault();
}
Later, i'm trying to do a query on a different DB against the list of Id's i pulled above (essentially a IN clause):
using (var db = new Entities.DB2.DB2Conn())
{
var accounts = db.Accounts.Where(mapping.MappingAccts.Any(y => y.Id == ?????????)).ToList();
}
As you can see i only got part way with this.
Basically what i need to do is query the Accounts table against it's ID column and pull all records that match mapping.MappingAccts.Id column.
Most of the examples i am finding explain nicely how to do this against a single dimension array but i'm looking to compare specific columns.
Any assist would be awesome.
Nugs

An IN clause is generated using a IEnumerable.Contains.
From the first DB1 context, materialize the list of Id's
var idList = mapping.MappingAccts.Select(m => m.Id).ToList();
Then in the second context query against the materialized list of id's
var accounts = db.Accounts
.Where(a => idList.Contains(a.Id))
.ToList();
The only problem you may have is with the amount of id's you are getting in the first list. You may hit a limit with the SQL query.

This will give the list of Accounts which have the Ids contained by MappingAccts
using (var db = new Entities.DB2.DB2Conn())
{
var accounts = db.Accounts.Where(s => mapping.MappingAccts.Any(y => y.Id == s.Id)).ToList();
}

Linq: let Count result into a specified column

I've got a table Installation which can contains one or many Equipements.
And for functionnal reasons, I've overwritten my table Installation and added a field NbrEquipements.
I want to fill this field with Linq, but I'm stuck...
Due to special reasons, there is no relation between these to tables. So, no Installation.Equipements member into my class. Therefore, no Installation.Equipements.Count...
I'm trying some stuff. Here is my code:
var query = RepoInstallation.AsQueryable();
// Some filter
query = query.Where(i => i.City.RegionId == pRegionId));
int?[] etatIds = { 2, 3 };
query = (from i in query
select new Installation
{
NbrEquipements= (from e in RepoEquipement.AsQueryable()
where e.InstallationSpecialId == i.SpecialId
&& (etatIds.Contains(e.EquEtat))
select e.SasId
).Count()
});
But with this try, I got this error:
The entity or complex type 'myModel.Installation' cannot be constructed in a LINQ to Entities query
I've tried some other stuff but I'm always turning around...
Another thing that can be useful for me: It would be great to fill a field called Equipements which is a List<Equipement>.
After that, I would be able to Count this list...
Is it possible ?
Tell me if I'm not clear.
Thanks in advance.
Here is the final code:
//In the class:
[Dependency]
public MyEntities MyEntities { get; set; }
//My Methode code:
var query = MyEntities .SasInstallations.AsQueryable();
// Some filter
query = query.Where(i => i.City.RegionId == pRegionId));
var liste = new List<Installation>();
var queryWithListEquipements =
from i in query
select new
{
Ins = i,
EquipementsTemp = (from eq in MyEntities.Equipements.AsQueryable()
where eq.SpecialId == i.SpecialId
&& (etatIds.Contains(eq.SasEquEtat))
select eq
).ToList()
};
var listWithListEquipements = queryWithListEquipements.ToList();
foreach (var anonymousItem in listWithListEquipements)
{
var ins = anonymousItem.Ins;
ins.Equipements = anonymousItem.EquipementsTemp;
ins.NumberEquipements = ins.Equipements.Count();
liste.Add(ins);
}
return liste;
By the way, this is very very fast (even the listing of Equipements). So this is working exactly has I wished. Thanks again for your help everyone!

Use an anonymous type. EF does not like to instantiate entity classes inside a query.
var results = (from i in query
select new
{
NbrEquipements= (from e in RepoEquipement
where e.InstallationSpecialId == i.SpecialId
&& (etatIds.Contains(e.EquEtat))
select e.SasId
).Count()
})
.ToList();
Notice how I used select new instead of select new Installation.
You can then use the data inside the list (which is now in memory) to create instances of type Installation if you want like this:
var installations = results.Select(x =>
new Installation
{
NbrEquipements = x.NbrEquipements
}).ToList();
Here is how to obtain the list of equipment for each installation entity:
var results = (from i in query
select new
{
Installation = i,
Equipment = (from e in RepoEquipement
where e.InstallationSpecialId == i.SpecialId
&& (etatIds.Contains(e.EquEtat))
select e).ToList()
})
.ToList();
This will return a list of anonymous objects. Each object will contain a property called Installation and another property called Equipment (which is a list). You can easily convert this list (of anonymous objects) to another list of whatever type that you want.

LINQ: Is there a way to combine these queries into one?

I have a database that contains 3 tables:
Phones
PhoneListings
PhoneConditions
PhoneListings has a FK from the Phones table(PhoneID), and a FK from the Phone Conditions table(conditionID)
I am working on a function that adds a Phone Listing to the user's cart, and returns all of the necessary information for the user. The phone make and model are contained in the PHONES table, and the details about the Condition are contained in the PhoneConditions table.
Currently I am using 3 queries to obtain all the neccesary information. Is there a way to combine all of this into one query?
public ActionResult phoneAdd(int listingID, int qty)
{
ShoppingBasket myBasket = new ShoppingBasket();
string BasketID = myBasket.GetBasketID(this.HttpContext);
var PhoneListingQuery = (from x in myDB.phoneListings
where x.phonelistingID == listingID
select x).Single();
var PhoneCondition = myDB.phoneConditions
.Where(x => x.conditionID == PhoneListingQuery.phonelistingID).Single();
var PhoneDataQuery = (from ph in myDB.Phones
where ph.PhoneID == PhoneListingQuery.phonePageID
select ph).SingleOrDefault();
}

You could project the result into an anonymous class, or a Tuple, or even a custom shaped entity in a single line, however the overall database performance might not be any better:
var phoneObjects = myDB.phoneListings
.Where(pl => pl.phonelistingID == listingID)
.Select(pl => new
{
PhoneListingQuery = pl,
PhoneCondition = myDB.phoneConditions
.Single(pc => pc.conditionID == pl.phonelistingID),
PhoneDataQuery = myDB.Phones
.SingleOrDefault(ph => ph.PhoneID == pl.phonePageID)
})
.Single();
// Access phoneObjects.PhoneListingQuery / PhoneCondition / PhoneDataQuery as needed
There are also slightly more compact overloads of the LINQ Single and SingleOrDefault extensions which take a predicate as a parameter, which will help reduce the code slightly.
Edit
As an alternative to multiple retrievals from the ORM DbContext, or doing explicit manual Joins, if you set up navigation relationships between entities in your model via the navigable join keys (usually the Foreign Keys in the underlying tables), you can specify the depth of fetch with an eager load, using Include:
var phoneListingWithAssociations = myDB.phoneListings
.Include(pl => pl.PhoneConditions)
.Include(pl => pl.Phones)
.Single(pl => pl.phonelistingID == listingID);
Which will return the entity graph in phoneListingWithAssociations
(Assuming foreign keys PhoneListing.phonePageID => Phones.phoneId and
PhoneCondition.conditionID => PhoneListing.phonelistingID)

You should be able to pull it all in one query with join, I think.
But as pointed out you might not achieve alot of speed from this, as you are just picking the first match and then moving on, not really doing any inner comparisons.
If you know there exist atleast one data point in each table then you might aswell pull all at the same time. if not then waiting with the "sub queries" is nice as done by StuartLC.
var Phone = (from a in myDB.phoneListings
join b in myDB.phoneConditions on a.phonelistingID equals b.conditionID
join c in ph in myDB.Phones on a.phonePageID equals c.PhoneID
where
a.phonelistingID == listingID
select new {
Listing = a,
Condition = b,
Data = c
}).FirstOrDefault();
FirstOrDefault because single throws error if there exists more than one element.

Inefficient entity framework queries

I have a following foreach statement:
foreach (var articleId in cleanArticlesIds)
{
var countArt = context.TrackingInformations.Where(x => x.ArticleId == articleId).Count();
articleDictionary.Add(articleId, countArt);
}
Database looks like this
TrackingInformation(Id, ArticleId --some stuff
Article(Id, --some stuff
what I want to do is to get all the article ids count from TrackingInformations Table.
For example:
ArticleId:1 Count:1
ArticleId:2 Count:8
ArticleId:3 Count:5
ArticleId:4 Count:0
so I can have a dictionary<articleId, count>
Context is the Entity Framework DbContext. The problem is that this solution works very slow (there are > 10k articles in db and they should rapidly grow)

Try next query to gather grouped data and them add missing information. You can try to skip Select clause, I don't know if EF can handle ToDictionary in good manner.
If you encounter Select n + 1 problem (huge amount of database requests), you can add ToList() step between Select and ToDictionary, so that all required information will be brought into memory.
This depends all your mapping configuration, environment, so in order to get good performance, you need to play a little bit with different queries. Main approach is to aggregate as much data as possible at database level with few queries.
var articleDictionary =
context.TrackingInformations.Where(trackInfo => cleanArticlesIds.Contains(trackInfo.ArticleId))
.GroupBy(trackInfo => trackInfo.ArticleId)
.Select(grp => new{grp.Key, Count = grp.Count()})
.ToDictionary(info => "ArticleId:" + info.Key,
info => info.Count);
foreach (var missingArticleId in cleanArticlesIds)
{
if(!articleDictionary.ContainsKey(missingArticleId))
articleDictionary.add(missingArticleId, 0);
}

If TrackingInformation is a navigatable property of Article, then you can do this:
var result=context.Article.Select(a=>new {a.id,Count=a.TrackingInformation.Count()});
Putting it into a dictionary is simple as well:
var result=context.Article
.Select(a=>new {a.id,Count=a.TrackingInformation.Count()})
.ToDictionary(a=>a.id,a=>a.Count);
If TrackingInforation isn't a navigatable property, then you can do:
var result=context.Article.GroupJoin(
context.TrackingInformation,
foo => foo.id,
bar => bar.id,
(x,y) => new { id = x.id, Count = y.Count() })
.ToDictionary(a=>a.id,a=>a.Count);