var query = from organization in _context.Organizations
join location in
(
from location in _context.Locations
orderby location.Name
select new
{
location.Id,
location.Name,
location.OrganizationId,
}
) on organization.Id equals location.OrganizationId into
locations
orderby organization.Name
select new
{
organization.Id,
organization.Name,
locations
};
I am using Entity Framework Core 1.1 . Here I am joining Organizations & Locations table in LINQ. But when I run the above LINQ query i get two separate select statements in SQL Profiler,
I see:
SELECT [organization].[Id], [organization].[Name]
FROM [Organization] AS [organization]
ORDER BY [organization].[Name]
SELECT [location].[Id], [location].[Name]
FROM [Location] AS [location]
ORDER BY [location].[LoName]
My expectation was a single query joining two tables in SQL Profiler. But what I am missing so this runs as a single join query in SQL?
EF Core 1.x cannot translate complex LINQ query to optimized TSQL query like EF 6. in such cases EF Core only generates query for fetching data from database and rest of operations like group by or complex where clause, will happen in memory. in your case generated tsql queries are for fetching data and join, order by and projection will happen in memory, so be careful if you have large data in those tables or write raw sql query instead of linq query.
Related
I have something like this in my C# MVC controller:
from table1 in db.Table1.AsQueryable()
join table2 in db.Table2.AsQueryable() on table1.Col1 equals table2.Col1
join table3 in db.table3.AsQueryable() on new { table2.Col2, table2.Col5 } equals new { table3.Col2, table3.Col5 }
.
.
few more joins
.
.
WHERE ......
select new {table1.Prop1, table2.Prop2, table3.Prop3}
When I watch what it runs on SQL profiler, I was expecting a single query with all the joins. What it does instead, it selects all columns from all tables in separate queries. i.e. Runs
SELECT * FROM Table2 --Instead of * it has all column names
when that's finished running, it runs
SELECT * FROM Table3 --Instead of * it has all column names
and so on for each table. Tables are big so it takes too long, using a lot of memory. I added AsQueryable() on the entities but it didn't make a difference, still multiple queries. db is a DbContext, using core 2.
How can I change the LINQ or some other setting so the whole thing runs as a single query?
Update
It looks like the problem was caused by having Convert.ToInt32( on one of the join columns. The int column I was joining on is nullable in one table and non-nullable in the other table, I had Convert.ToInt32( on the nullable
table, removing the convert generated a single query.
According to LINQ2SQL documents:
When you query for an object, you actually retrieve only the object you requested. The related objects are not automatically fetched at the same time.
The DataLoadOptions class provides two methods to achieve immediate loading of specified related data. The LoadWith method allows for immediate loading of data related to the main target. The AssociateWith method allows for filtering related objects.
This is an issue with Lazy Loading vs Eager Loading.
This is a great post with very good explanation.
Lazy Loading And Eager Loading In LINQ To SQL
I have this join :
var andlist = (from cust in custFinal
join serv in db.Service on cust.ID equals serv.CustID
select new JoinObj
{
Name = cust.name,
ServiceID = serv.ID,
});
custFinal is a list of Customers that contains only one object. db.Service is a DbSet and there are only four rows whose custID equals customer object's ID in Service table. When I use ToList() or Count(), used memory quickly exceeds 1GB and I'm getting outOfMemory exception. Can you tell me what is wrong with this code? Thanks in advance.
The reason is you don't really perform join on server. custFinal as you said is just in-memory list, not a database table or query. So it is IEnumerable, not IQueryable. When you perform a join - it calls IEnumerable.Join, not IQueryable.Join method. The latter would build a query but the former will just pull all arguments into memory and perform join in-memory. So in result - whole Service table in pulled into memory and joined there (easy to check if you log EF context queries - you will see that it just performs select all from Service query).
If you change the order of arguments in a join so that IQueryable.Join would be executed - that won't help either, because you cannot join database table with in-memory list with Entity Framework anyway. So you have to find another way, for example:
var ids = custFinal.Select(c => c.ID).ToArray();
var matchingServices = db.Service.Where(serv => ids.Contains(serv.CustID)).Select(c => new {c.ServiceID, c.CustID}).ToArray();
// now filter `custFinal` based on `matchingServices`, in memory.
That will perform CustID IN (...) query instead of a join. If you insist on having a join - you will have to do that with raw sql, without entity framework (you will also need to create custom table type in sql server, if you use SQL server).
This is my first time using EF 6 as well as MySQL. I came across an annoyance while updating my LINQ statement from explicitly using joins to using navigation properties to fetch related data.
Here is the statement I'm executing to get a user and all the user's locations.
AspNetUsers.Include("UserLocations")
.Select(u => new {
FullName = u.FullName,
Locations = u.UserLocations.Select(l => l.Title)
})
This statement, using LinqPad4, generates the following SQL:
Why does it join using a select statement instead of doing a join on the table itself, and why does it add all the location columns to the join when the only column needed is the Title?
Wouldn't the following SQL query be better:
SELECT
u.FullName,
l.Title
FROM AspNetUsers u
JOIN UserLocations ul ON u.Id = ul.UserId
JOIN Locations l ON ul.LocationId = l.LocationId;
This is my first time using EF, I have read that in the past the SQL generated has not been so great. I was wandering if this is just one of those cases or if there is something I could do to minimize the SQL generated.
Thank you in advance!
Union with NHibernate and Criteria:
Is it possible in Criteria or QueryOver?
If not, is there any other way to achieve a union of two result within the same query?
You can't do a union directly, but you can do two future queries and union the results in code:
var resultSet1 = this.Session.CreateCriteria<A>().Future<A>();
var resultSet2 = this.Session.CreateCriteria<B>().Future<B>();
After this, when either result set is enumerated, NHibernate will issue a single query to the database which will return multiple result sets. Note, if you are not using SQL Server, the database may not support multiple result sets.
This is not possible even using HQL. See this other S.O. post
One way is to drop back to raw SQL and use a named query
<sql-query name="MyQuery">
<![CDATA[
select col1,col2 from table1
union
select col1,colA from table2
]]>
</sql-query>
And use the AliasToBeanResultTransformer to transform it back into your DTO/POCO
var query = Session
.GetNamedQuery("MyQuery")
.SetResultTransformer(new AliasToBeanResultTransformer(typeof(MyDto)));
return query.List<MyDto>();
You can use -
NHibernate.Criterion.Restrictions.Or(ICriterion FirstQuery,
ICriterion SecondQuery)
as your Criteria in a single query.
I've got a scenario where I will need to order by on a column which is a navigation property for the Users entity inside my EF model.
The entities:
Users --> Countries 1:n relationship
A simple SQL query would be as follows:
SELECT UserId, u.Name, c.Name
FROM users u join countries c on u.CountryId = c.CountryId
ORDER BY c.Name asc;
So then I tried to replicate the above SQL query using Linq to Entities as follows - (Lazy Loading is enabled)
entities.users.OrderBy(field => field.country.Name).ToList();
But this query does not return my countries sorted by their name as the native SQL query above does.
However I continued a bit more and did the following:
var enumeratedUsers = entities.users.AsEnumerable();
users = enumeratedUsers.OrderBy(fields => fields.country.Name).ToList();
But ordering on the enumeratedUser object for about 50 records took approx. 7seconds
Is there a better way how to omit the Enumerable and without returning an anonymous type?
Thanks
EDIT
I just forgot to say that the EF provider is a MySQL one not a MS SQL. In fact I just tried the same query on a replicated database in MS SQL and the query works fine i.e. the country name is ordered correctly, so it looks like I have no other option apart from getting the result set from MySQL and execute the order by from the memory on the enumerable object
var enumeratedUsers = entities.users.AsEnumerable();
users = enumeratedUsers.OrderBy(fields => fields.country.Name).ToList();
This is LINQ to Objects not LINQ to Entities.
Above Order By clause will call OrderBy defined in Enumerable
That is ordering will be done in memory. Hence it will take long time
Edit
It looks like a MySQL related issue
You may try something like this.
var users = from user in entities.users
join country in entities.Country on user.CountryId equals country.Id
orderby country.Name
select user;
entities.users.OrderBy(field => field.country.Name).ToList();
But this query does not return my countries sorted by their name as the native
SQL query above does.
Yes, it does not return Countries but only Users sorted by the name of country.
When this query is executed, the following sql is sent to DB.
SELECT u.*
FROM users u join countries c on u.CountryId = c.CountryId
ORDER BY c.Name asc;
As you can see, the result does not include any fields of countries. As you mentioned the lazy loading, countires are loaded through it when needed. At this time, countries are ordered as the order you call it through the lazy loading. You can access countries through the Local property of a entity set.
This point tells you that if you want user sorted by the name of country and also countires sorted by the name, you need the eagerly loading as #Dennis mentioned like:
entities.users.Include["country"].OrderBy(field => field.country.Name).ToList();
This is converted to the following sql.
SELECT u.*, c.*
FROM users u join countries c on u.CountryId = c.CountryId
ORDER BY c.Name asc;
Have you tried using Include?
entities.users.Include["country"].OrderBy(field => field.country.Name).ToList();
SOLUTION
Since I had both columns named Name in both Countries and Users table MySQL Connector was generating this output when order by country.Name was executed:
SELECT `Extent1`.`Username`, `Extent1`.`Name`, `Extent1`.`Surname`, `Extent1`.`CountryId`
FROM `users` AS `Extent1` INNER JOIN `countries` AS `Extent2` ON `Extent1`.`CountryId` = `Extent2`.`CountryId`
ORDER BY `Name` ASC
therefore this will result in ordering on the users.Name rather countries.Name
However MySQL have release version 6.4.3 .NET connector which has resolved a bunch of issues one of them being:
We are also including some SQL generation improvements related to our entity framework provider. Source: http://forums.mysql.com/read.php?3,425992
Thank you for all your input. I tried to be clear as much as possible to help others which might encounter my same issue.