I am writing a LINQ query similar to the following:
var test = from o in dbcontext.Orders.AsEnumerable()
where o.ID==1
select new Order
{
Name = GetName(o.ID)
};
In order to call an external function within the LINQ query, I am using AsEnumerable() within the query.
I understand that normally the query is not executed until an enumeration function like ToList(), etc. is called. But here I am calling the enumeration within the query.
Can anyone tell me if it is considered bad practice to use AsEnumerable() like this? Will it suffer in performance compared to calling ToList() after creating the query?
I'd do
var ids = from o in dbcontext.Orders
where o.ID==1
select new { ID = o.ID };
var names = from i in ids.AsEnumerable()
select new Order { Name = GetName(i.ID) };
i.e. do as much querying as possible in the database, and then only perform the ID-to-name transformation in C#.
Related
Trying to implement a correct Repository pattern with Entity Framework, I'm stumbling over some issues with let statements. What I want to do is:
var customer = (from cus in Customers.GetAll()
let brokerExists = InsuredBrokers.GetAll().Any(ib => ib.INS_Id == cus.INS_Id)
// ... more stuff
But this will give me an error
System.NotSupportedException: 'LINQ to Entities does not recognize the
method
'System.Linq.IQueryable`1[SNIP.DataModel.EA_INB_InsuredBrokers_TB]
GetAll()' method, and this method cannot be translated into a store
expression.'
What I instead can do is:
var customer = (from cus in Customers.GetAll()
let brokerExists = _context.Set<EA_INB_InsuredBrokers_TB>().Any(ib => ib.INS_Id == cus.INS_Id)
// ... more stuff
However, this breaks any point in using the Repository pattern. When I search for answers, people say to put it in a query on its own and reference it from memory, but since I actually have the customer's Id (INS_Id) in the let statement, I cannot do that.
GetAll() is like:
public IQueryable<T> GetAll()
{
return _context.Set<T>().AsQueryable();
}
Are there any clever ways to get around this?
You have to move InsuredBrokers.GetAll() out of the query:
var allBrokers = InsuredBrokers.GetAll();
var customer = (from cus in Customers.GetAll()
let brokerExists = allBrokers.Any(ib => ib.INS_Id == cus.INS_Id)
// ... more stuff
Then it will work fine. Since GetAll returns IQueryable and you don't enumerate it - this has no negative effect and there will still be one query to database, just like in your example with _context.
The reason is let statement is compiled like this:
Customers.GetAll().Select(cus => new {cus, brokerExists = InsuredBrokers.GetAll().Any(ib => ib.INS_Id == cus.INS_Id)}
Which means your call to InsuredBrokers.GetAll() is part of expression tree (it's inside Select expression), and entity framework cannot (will not) just call it to obtain value. It will try to translate it to SQL query, but has no idea what to do with GetAll method.
I assume you are trying to do a left join.
Your query:
var query = from customer in Customers
join broker in InsuredBrokers
on customer.InsertId equals broker.InsertId
into resutGroups
select new { Name = broker.Name, InsertId= broker.InsertId};
If you need entries for no maches use DefaultIfEmpty.
You can refer to this for more on the same.
Why does one method work but not the other, when it seems they are both doing the same thing, i.e. constructing an entity. My question then, is there a way to construct the entity in a L2E query instead of having to use just Linq or indeed both?
This works fine...
var queryToList = (from ac in ctx.AuthorisationChecks
where wedNumbers.Contains(ac.WedNo)
orderby ac.WedNo, ac.ExpAuthDate, ac.ActAuthDate
select new AuthorisationCheck
{
Blah = ac.Blah
}).ToList();
model.AuthorisationChecks = queryToList.Select(x => new AuthorisationCheck
{
Blah = x.Blah
}).ToList();
However, if i change...
var queryToList
to
model.AuthorisationChecks queryToList // Of type List<AuthorisationCheck>
i get the error in the Title...
The entity or complex type 'Model.AuthorisationCheck' cannot be constructed in a LINQ to Entities query.
EDIT:
In the model it is simply, nothing fancy here.
public List<AuthorisationCheck> AuthorisationChecks { get; set; }
EDIT2:
Tidied this up a little to be (which works fine)...
model.AuthorisationChecks = (from ac in ctx.AuthorisationChecks
where wedNumbers.Contains(ac.WedNo)
orderby ac.WedNo, ac.ExpAuthDate, ac.ActAuthDate
select ac).ToList()
.Select(x => new AuthorisationCheck
{
Blah = x.Blah
}).ToList();
EDIT2: My Solution
I wasn't happy with the anonymous type method and so went ahead and created a simple model containing only the properties I required to be used in the viewmodel.
Changed the type of model.AuthorisationChecks
from
List<AuthorisationCheck> // List of Entities
to
List<AuthorisationCheckModel> // List of models
which allows the following code to work, and without profiling it seems a lot quicker than using an anonymous type (and of course I don't cast to a list twice!).
model.AuthorisationChecks = (from ac in ctx.AuthorisationChecks
where wedNumbers.Contains(ac.WedNo)
orderby ac.WedNo, ac.ExpAuthDate, ac.ActAuthDate
select new AuthorisationCheckModel
{
Blah = x.Blah
}).ToList();
P.S. I was once warned by a coworker (who used to work for Microsoft) that it isn't a good idea to directly use Entities in this manner, maybe this was one of the reasons he was thinking of, I've also noticed some odd behavior using Entities directly in other cases (mainly corruptions).
Note: I generally use lambda expression instead of Fluent API, but the root issue should be the same.
I have historically noticed LINQ is unable to use C# classes for Select statements if the original datasource (i.e. ctx for you) is accessed by translating your query into SQL.
In other words, there are issues when getting something from the database and casting it to a custom class in the same chain.
LINQ is smart enough that it actually doesn't immediately execute your chained calls. It simply internally builds a query, and when you actually access your results (i.e. retrieve the value from memory), it executes the query.
I assume this is also the reason why you are faced with this error, because LINQ translates everything (including the Select) to SQL, and fails because there's no SQL-way to express it. In short, LINQ can't do a built query half-SQL, half-code. It's either all in SQL, or all in code.
You can usually confirm that this is the case by first making a List<> of the database table, then run the exact same query on it.
var myTable = db.AuthorizationCheck.ToList();
var myResult = myTable. //query here
Note: That is not a solution!
Taking the entire table in memory is an overly intensive way to work around this. It just proves the point that the problem isn't encountered if the datasource is in memory, but the error does occur if it's in a database.
There are ways I've fixed this, although I've never found a uniform way to approach this problem (generally depends on the opinion of my code reviewer, whether he likes the fix or not)
Using anonymous types, you can select what you want, and later cast it to the correct class. You can use the exact same fields, making a later cast easier to understand.
//Simpler query for clarity's sake
var myAnonymousResult = ctx.AuthorizationChecks
.Where(x => x.IsActive)
.Select(x => new { Name = x.Name, IsActive = x.IsActive })
.ToList();
var myCastResult = myAnonymousResult.Select(x => new Check() { Name = x.Name, IsActive = x.IsActive }).ToList();
If you use lambda expressions instead of the fluent API, you can call .ToList() after applying the filters but before calling the .Select() method. This ensures the current query will be executed, retrieved from the database, and put into an actual List<> in memory. At that point, you can call the .Select() statement without running into the same problem.
//Simpler query for clarity's sake
var myCastResult = ctx.AuthorizationChecks
.Where(x => x.IsActive)
.ToList()
.Select(x => new Check() { Name = x.Name, IsActive = x.IsActive });
Unfortunately though, my experience with this problem is anecdotal. I've never been able to officially confirm my suspicions as to the root cause of this issue; but the workarounds I mentioned should work as I've applied them numerous times in the past.
If anyone has an explanation of the root cause, I'd be very interested in hearing it!
If this query works fine:
var queryToList = (from ac in ctx.AuthorisationChecks
where wedNumbers.Contains(ac.WedNo)
orderby ac.WedNo, ac.ExpAuthDate, ac.ActAuthDate
select new AuthorisationCheck
{
Blah = ac.Blah
}).ToList();
then this should also work:
model.AuthorisationChecks = (from ac in ctx.AuthorisationChecks
where wedNumbers.Contains(ac.WedNo)
orderby ac.WedNo, ac.ExpAuthDate, ac.ActAuthDate
select new AuthorisationCheck
{
Blah = ac.Blah
}).ToList();
and in your first case you don't need to project again, you can directly assign it to model propeerty:
model.AuthorisationChecks = queryToList;
UPDATE:
As it is Linq To Entities,you have to do something like this using anonymous type:
var queryToList = (from ac in ctx.AuthorisationChecks
where wedNumbers.Contains(ac.WedNo)
orderby ac.WedNo, ac.ExpAuthDate, ac.ActAuthDate
select new
{
Blah = ac.Blah
}).ToList();
and then:
model.AuthorisationChecks = queryToList.Select(x => new AuthorisationCheck
{
Blah = x.Blah
}).ToList();
The database contains Orders.
Orders can be contained within a group of Orders.
For every group of Orders it could contain 1 to many Orders.
However, Orders could have a NULL value assigned GroupOrderId as previous Orders did not have the grouping concept. Only new Orders enforce the concept of being added to a group.
The class structure to be populated in order to perform actions on each Order is
public class OrdersGroup
{
public int? GroupOrderId { get; set; }
public List<int> OrderIds { get; set; }
}
The linq statement
var workPacketOrdersList = (from o in db.Orders
where
o.GroupOrderId >= groupOrderIdMin && o.GroupOrderId <= groupOrderIdMax &&
o.IsDeleted == false
orderby o.WorkPacketId ascending
group o by o.WorkPacketId
into grp
select new OrdersGroup
{
GroupOrderId = grp.Key,
OrderIds = grp.Select(g => g.OrderId).ToList()
}).ToList();
Full exception
LINQ to Entities does not recognize the method 'System.Collections.Generic.List`1[System.Int32] ToList[Int32](System.Collections.Generic.IEnumerable`1[System.Int32])' method, and this method cannot be translated into a store expression.
I see that the returned type of the linq query is a List<OrdersGroup>.
If the final .ToList() is omitted from the query than the return type becomes an IQueryable<OrdersGroup>
No matter what action is performed next the result is an exception that this method cannot be translated into a store expression.
I have tried to remove the specific select new OrdersGroup into a more generic select new and then perform actions on this result only to find the same store expression exception.
Can someone give some insight into where this linq is incorrect?
this is the part that's failing - grp.Select(g => g.OrderId).ToList() - you can't have a .ToList() in the select clause. remove that and you should be fine.
The problem is that LINQ to Entities is attempting to convert your query into SQL. It doesn't know how translate ToList into SQL, so that's the problem. You need to remove the call to ToList from inside your query.
That is,
OrderIds = grp.Select(g => g.OrderId).ToList()
LINQ to Entities can not convert that to SQL. Remove the call
OrderIds = grp.Select(g => g.OrderId)
and if you need OrderIds to be a List<int>, do the call to ToList after you execute the query.
It's because you're trying to call ToList() in a part of the query that will become raw SQL and executed at the source (ie SQL Server, not the CLR). I don't know exactly what your data is so I can't necessarily make an accurate recommendation on how to fix it but I would try taking making the ToList() call after this query or just not making it all. It's likely IEnumberable will offer whatever functionality you need which is what the Select will return if you remove the ToList() call.
By the way since I wasn't explicit, I'm referring to the ToList() call inside the select -(second to last line) OrderIds = grp.Select(g => g.OrderId).ToList() the other one is fine. It's executed on the results of the SQL query which is perfectly fine, you just can't make calls to C# specific methods within a query that will be executed by the SQL provider.
Your problem is that you select a list in your select statement.
select new OrdersGroup
{
GroupOrderId = grp.Key,
OrderIds = grp.Select(g => g.OrderId).ToList()
/////////////////////////////////////^^^^^^^^^HERE
}
What you need to do is change OrderIds to an IEnumerable<int>, and then get rid of the ToList.
I have a sql statement like this:
DECLARE #destinations table(destinationId int)
INSERT INTO #destinations
VALUES (414),(416)
SELECT *
FROM GroupOrder grp (NOLOCK)
JOIN DestinationGroupItem destItem (NOLOCK)
ON destItem.GroupOrderId = grp.GroupOrderId
JOIN #destinations dests
ON destItem.DestinationId = dests.destinationId
WHERE OrderId = 5662
I am using entity framework and I am having a hard time getting this query into Linq. (The only reason I wrote the query above was to help me conceptualize what I was looking for.)
I have an IQueryable of GroupOrder entities and a List of integers that are my destinations.
After looking at this I realize that I can probably just do two joins (like my SQL query) and get to what I want.
But it seems a bit odd to do that because a GroupOrder object already has a list of DestinationGroupItem objects on it.
I am a bit confused how to use the Navigation property on the GroupOrder when I have an IQueryable listing of GroupOrders.
Also, if possible, I would like to do this in one trip to the database. (I think I could do a few foreach loops to get this done, but it would not be as efficient as a single IQueryable run to the database.)
NOTE: I prefer fluent linq syntax over the query linq syntax. But beggars can't be choosers so I will take whatever I can get.
If you already have the DestinationGroupItem as a Navigation-property, then you already have your SQL-JOIN equivalent - example. Load the related entities with Include. Use List's Contains extension method to see if the desired DestinationId(s) is(are) hit:
var destinations = new List<int> { 414, 416 };
var query = from order in GroupOrder.Include(o => o.DestinationGroupItem) // this is the join via the navigation property
where order.OrderId == 5662 && destinations.Contain(order.DestinationGroupItem.DestinationId)
select order;
// OR
var query = dataContext.GroupOrder
.Include(o => o.DestinationGroupItem)
.Where(order => order.OrderId == 5662 && destinations.Contain(order.DestinationGroupItem.DestinationId));
I have seen these StackOverflow Answers but they do not produce the same results when the filtering list is in memory.
I have a list of Ids. I want to remove any IDs that exists in the database, so that I am left with a list of IDs that need to be added. In other words, I need to perform a where not in SQL query, using Linq-To-Entities. The problem is, instead of producing that SQL, these methods each produce a SQL query per list item.
var providerIds = new [] {"1130", "1", "16"};
Method 1:
var missingProviders = (from provider in providerIds
where !JobProviders.Any(p => p.JobProviderID == provider)
select provider).ToList();
Method 2:
var missingProviders = (from provider in providerIds
where !(from p in JobProviders select p.JobProviderID).Contains(provider)
select provider).ToList();
Is there a way to structure the LINQ query such that it produces the intended not in form, or are these the only solutions?
What about something like
var providersInDb = (from provider in JobProviders
where providerIds.Contains(provider.JobProviderID)
select provider.JobProviderID).ToList();
var missingProviders = providerIds.Where(p => !providersInDb.Contains(p))
Tricky. I don't have my tools in front of me, so I don't know how this will pan out exactly.
var dbProviderIds = JobProviders.Select(p => p.JobProviderId);
var allProviders = dbProviderIds.Union(providerIds).Distinct();
var missing = allProviders.Except(dbProviderIds);
On the DB, get all the provider Ids, then combine that with the in-memory ones. Then remove the ones that are known on the database.