Hi all I am having a sql query which I would like to execute the same using entity framework and bind it to gridview can some one help me. This is my query
SELECT Technology,[Description], Technologies.TechID, COUNT(*) AS 'num_employees'
FROM Technologies
LEFT JOIN Questions
ON Questions.TechID = Technologies.TechID
GROUP BY Technologies.TechID, Technology,[Description]
How can I convert the same to get the results using EF
Personally, I would make this a stored procedure instead, and import it into EF. Then you would just call it in your code and databind the result to your grid.
This way the joining, grouping, etc all happens server side instead of client side. Otherwise, you'd be sending a lot more information than necessary to the client. Plus, the server can theoretically do these operations faster than a client's.
Try this, i didn't have the time to test it both for logic and syntax. You need something similar.
var q = from a in context.Technologies
join b in context.Questions
on a.TechID equal b.TechID into j1
from j2 in j1.DefaultIfEmpty()
group j2 by new { a.TechID, b.Technology, b.Description into grouped
select new { Tecnology = grouped.Technology, Description = grouped.Description, Count = grouped.Count() }
Related
We have 18 table join which is typical for ERP systems. The join is done via LINQ over Entity Framework.
The join gets progressively slower as more joins are added. The return result set is small(15 records). The LINQ generated query is captured via SQL Profiler and when we run this via Microsoft Management Console it is very fast : 10ms. When we run it via our C# LINQ-over-EntityFramework it takes 4 seconds.
What i guess is happening:
The time it takes to compile expression tree into SQL is 2 seconds out of total 4 seconds, and another 2 seconds i guess is spent internally to convert SQL result set into actually C# classes. Also it is not connected to initialization of entity framework because we run some queries before and repetitive calls to this join produce same 4 seconds.
Is there a way to speed this up. Otherwise we are considering abandoning Entity Framework for being absolutely inefficient...
In case it helps, I had a nasty performance issue, whereby a simple query that took 1-2 seconds in raw SQL took about 11 seconds via EF.
I went from using...
List<GeographicalLocation> geographicalLocations = new SalesTrackerCRMEntities()
.CreateObjectSet<GeographicalLocation>()
.Where(g => g.Active)
.ToList();
which took about 11 seconds via EF, to using...
var geographicalLocations = getContext().CreateObjectSet<GeographicalLocation>()
.AsNoTracking()
.Where(g => g.Active).ToList();
which took less than 200 milliseconds.
The disadvantage to this is that it won't load related entities, so you have to load them manually afterwards, but it gives such an immense performance boost that it was well worth it (in this case at least).
You would have to assess each case individually to see if the extra speed is worth the extra code.
You correctly identified bottlenecks.
If you have quite complex queries, I would suggest you to use compiled queries to overcome expression tree to sql query conversion.
You can refer Compiled Queries in EF from here.
Fo second part if EF is using two much time materialize your object graph then I would suggest to use some other means to retrieve data apart from EF.
One option can be Dapper.NET, You can have your concise sql query and you can directly retrieve its result in concrete model objects using Dapper (or any other tiny ORM)
I suspect your query takes so long to generate becuase you are treating Entity Framework like it is a SQL Query, which is not correct. You have many joins and akward calls in your linq syntax. Generally, your syntax should be similar to the following fictitious modeling query:
var result = (from appointment in appointments
from operation in appointment.Operations
where appointment.Id == 12
select new Model {
Id = appointment.Id,
Name = appointment.Name,
// etc, etc
}).ToList();
There is no use of joins above, the navigation property between Appointment and Operations takes care of the neccessary plumbing. Remember, this is an ORM, there is no concept of a join, only a concept of relationships.
The call to Distinct at the end, also indicates the structure of the db schema may be problematic if it returns too many duplicate results.
If after refactoring the entity model and correctly constructing the query still leaves with underperformance, it is advisable to use a stored procedure and map the result with EF's built in methods for doing so.
It is hard to tell what is going wrong here without seeing how you are using linq, but I suspect this will fix your problem:
var myResult = dataContext.table.Where(x == "Your joins and otherstuff").ToList();
//after converting it to a list use it how you need, but not before.
If this does not help please post your code.
The problem is that you are probably passing it to a data source that is running all sorts of additional queries based on you open result set.
Try this instead:
IEnumerable<SigmaTEK.Web.Models.SchedulerGridModel> tasks = (from appointment in _appointmentRep.Value.Get().Where(a => (a.Start < DbContext.MaxTime && DbContext.MinTime < a.Expiration))
join timeApplink in _timelineAppointmentLink.Value.Get().Where(a => a.AppointmentId != Guid.Empty)
on appointment.Id equals timeApplink.AppointmentId
join timeline in timelineRep.Value.Get().Where(i => timelines.Contains(i.Id))
on timeApplink.TimelineId equals timeline.Id
join repeater in _appointmentRepeaterRep.Value.Get().Where(repeater => (repeater.Start < DbContext.MaxTime && DbContext.MinTime < repeater.Expiration))
on appointment.Id equals repeater.Appointment
into repeaters
from repeater in repeaters.DefaultIfEmpty()
join aInstance in _appointmentInstanceRep.Value.Get()
on appointment.Id equals aInstance.Appointment
into instances
from instance in instances.DefaultIfEmpty()
join opRes in opResRep.Get()
on instance.ResourceOwner equals opRes.Id
into opResources
from op in opResources.DefaultIfEmpty()
join itemResource in _opDocItemResourcelinkRep.Value.Get()
on op.Id equals itemResource.Resource
into itemsResources
from itemresource in itemsResources.DefaultIfEmpty()
join opDocItem in opDocItemRep.Get()
on itemresource.OpDocItem equals opDocItem.Id
into opDocItems
from opdocitem in opDocItems.DefaultIfEmpty()
join opDocSection in opDocOpSecRep.Get()
on opdocitem.SectionId equals opDocSection.Id
into sections
from section in sections.DefaultIfEmpty()
join opDoc in opDocRep.Get()
on section.InternalOperationalDocument equals opDoc.Id
into opdocs
from opdocitem2 in opDocItems.DefaultIfEmpty()
join opDocItemLink in opDocItemStrRep.Get()
on opdocitem2.Id equals opDocItemLink.Parent
into opDocItemLinks
from link in opDocItemLinks.DefaultIfEmpty()
join finItem in finItemsRep.Get()
on link.Child equals finItem.Id
into temp1
from rd1 in temp1.DefaultIfEmpty()
join sec in finSectionRep.Get()
on rd1.SectionId equals sec.Id
into opdocsections
from finopdocsec in opdocsections.DefaultIfEmpty()
join finopdoc in opDocRep.Get().Where(i => i.DocumentType == "Quote")
on finopdocsec.InternalOperationalDocument equals finopdoc.Id
into finOpdocs
from finOpDoc in finOpdocs.DefaultIfEmpty()
join entry in entryRep.Get()
on rd1.Transaction equals entry.Transaction
into entries
from entry2 in entries.DefaultIfEmpty()
join resproduct in resprosductRep.Get()
on entry2.Id equals resproduct.Entry
into resproductlinks
from resprlink in resproductlinks.DefaultIfEmpty()
join res in resRep.Get()
on resprlink.Resource equals res.Id
into rootResource
from finopdoc in finOpdocs.DefaultIfEmpty()
join rel in orgDocIndRep.Get().Where(i => (i.Relationship == "OrderedBy"))
on finopdoc.Id equals rel.OperationalDocument
into orgDocIndLinks
from orgopdoclink in orgDocIndLinks.DefaultIfEmpty()
join org in orgRep.Get()
on orgopdoclink.Organization equals org.Id
into toorgs
from opdoc in opdocs.DefaultIfEmpty()
from rootresource in rootResource.DefaultIfEmpty()
from toorg in toorgs.DefaultIfEmpty()
select new SigmaTEK.Web.Models.SchedulerGridModel()
{
Id = appointment.Id,
Description = appointment.Description,
End = appointment.Expiration,
Start = appointment.Start,
OperationDisplayId = op.DisplayId,
OperationName = op.Name,
AppContextId = _appContext.Id,
TimelineId = timeline.Id,
AssemblyDisplayId = rootresource.DisplayId,
//Duration = SigmaTEK.Models.App.Utils.StringHelpers.TimeSpanToString((appointment.Expiration - appointment.Start)),
WorkOrder = opdoc.DisplayId,
Organization = toorg.Name
}).Distinct().ToList();
//In your UI
MyGrid.DataSource = tasks;
MyGrid.DataBind();
//Do not use an ObjectDataSource! It makes too many extra calls
First of all I'm sorry because this is the second time that I write this question but before was bad explained and now is close.
I'm doing a linq query for a search page for a CRM data base, and wrtiting a normal query like below is not working, I'm getting the exception:
[System.ServiceModel.FaultException<Microsoft.Xrm.Sdk.OrganizationServiceFault>] = {"'Contact' entity doesn't contain attribute with Name = 'title'."}
For a join query, that in the clause Where was something like r.Name == "Me" && j.LastName == "He" I had to did the query with two Where clauses, because I was getting the same exception as above, saying that table 'r' doesn't have 'LastName' attributte.
var cms = from i in aux_pr
join cal in Contact on i.aux_CallerRequestorID.Id equals cal.ContactId.Value
join sub in Subject on i.aux_ClassificationID.Id equals sub.SubjectId
where cal.FullName.Contains(searchTerm) ||
sub.Title.Contains(searchTerm)
In this case, how can I do this query. Thanks in advance!
I want to comment what have I learned and the solution that I have found to my problem hoping could help some one. There are some limitations in CRM LINQ, as explained here
The first that I found, having an entity reference like this:
CrmEntityReference Caller
{
Guid ID;
string name;
}
I can select Caller.name but I CAN'T have Caller.name in the where clause. Solution for this -> Join the table
The second limitation, is when we have joins in the query, we can have different tables in the where if they are an AND predicate, we have to write two clauses where like this:
where cal.FullName.Contains(searchTerm)
where sub.Title.Contains(searchTerm)
But the problem comes when instead of an AND we need use an OR predicate, the only solution we have is do two queries and after do an Union of these queries.
I have four queries for a call that could be done just with one, now in developing stage performance is good due to the amount of records, but we'll see in testing stage how this work.
try to create two different filters..
var cms = from i in aux_pr
join cal in Contact on i.aux_CallerRequestorID.Id equals cal.ContactId.Value
join sub in Subject on i.aux_ClassificationID.Id equals sub.SubjectId
where cal.FullName.Contains(searchTerm) ||
where sub.Title.Contains(searchTerm)
I'm trying to convert this query
select top(10) *
from SOMETABLE
where Name = 'test'
into linq so i think it should look like this
var c =
(from l
in db.SOMETABLE
where l.Name= 'test'
select l).take(10);
But when I look into server profiler I can see that linq takes all the data from table and probably apply WHERE and TAKE afer pooling data from database.
The problem is that the SOMETABLE have ~10 000 000 records and it does not work fast.
Am I doing it wrong?
The code you posted has at least 3 mistakes, so I assume it isn't your actual code. To get the symptom you describe, the most likely cause is that you have used IEnumerable<T> somewhere, and are composing from that. To get end-to-end query composition (i.e. to do the TOP at the database), you need to be using IQueryable<T>. For example, the following is broken:
IEnumerable<SomeType> data = db.SomeTable;
var c = (from l in data
where l.Name == "test"
select l).Take(10);
but the following is absolutely fine, noting that only the first line has changed:
IQueryable<SomeType> data = db.SomeTable;
var c = (from l in data
where l.Name == "test"
select l).Take(10);
noting that this is also identical to:
IQueryable<SomeType> data = db.SomeTable;
var c = data.Where(l => l.Name == "test").Take(10);
So: make sure you haven't forced it to IEnumerable<T> (or similar, such as lists) prematurely.
As a final note, IIRC Entity Framework demands an ordering if you are applying skip/take (erroring if you don't) - this further supports my guess that you have dropped to IEnumerable<T> too early, but: don't be amazed if you need to specifcy an order by too.
I have a situation where my application constructs a dynamic LINQ query using PredicateBuilder based on user-specified filter criteria (aside: check out this link for the best EF PredicateBuilder implementation). The problem is that this query usually takes a long time to run and I need the results of this query to perform other queries (i.e., joining the results with other tables). If I were writing T-SQL, I'd put the results of the first query into a temporary table or a table variable and then write my other queries around that. I thought of getting a list of IDs (e.g., List<Int32> query1IDs) from the first query and then doing something like this:
var query2 = DbContext.TableName.Where(x => query1IDs.Contains(x.ID))
This will work in theory; however, the number of IDs in query1IDs can be in the hundreds or thousands (and the LINQ expression x => query1IDs.Contains(x.ID) gets translated into a T-SQL "IN" statement, which is bad for obvious reasons) and the number of rows in TableName is in the millions. Does anyone have any suggestions as to the best way to deal with this kind of situation?
Edit 1: Additional clarification as to what I'm doing.
Okay, I'm constructing my first query (query1) which just contains the IDs that I'm interested in. Basically, I'm going to use query1 to "filter" other tables. Note: I am not using a ToList() at the end of the LINQ statement---the query is not executed at this time and no results are sent to the client:
var query1 = DbContext.TableName1.Where(ComplexFilterLogic).Select(x => x.ID)
Then I take query1 and use it to filter another table (TableName2). I now put ToList() at the end of this statement because I want to execute it and bring the results to the client:
var query2 = (from a in DbContext.TableName2 join b in query1 on a.ID equals b.ID select new { b.Column1, b.column2, b.column3,...,b.columnM }).ToList();
Then I take query1 and re-use it to filter yet another table (TableName3), execute it and bring the results to the client:
var query3 = (from a in DbContext.TableName3 join b in query1 on a.ID equals b.ID select new { b.Column1, b.column2, b.column3,...,b.columnM }).ToList();
I can keep doing this for as many queries as I like:
var queryN = (from a in DbContext.TableNameN join b in query1 on a.ID equals b.ID select new { b.Column1, b.column2, b.column3,...,b.columnM }).ToList();
The Problem: query1 is takes a long time to execute. When I execute query2, query3...queryN, query1 is being executed (N-1) times...this is not a very efficient way of doing things (especially since query1 isn't changing). As I said before, if I were writing T-SQL, I would put the result of query1 into a temporary table and then use that table in the subsequent queries.
Edit 2:
I'm going to give the credit for answering this question to Albin Sunnanbo for his comment:
When I had similar problems with a heavy query that I wanted to reuse in several other queries I always went back to the solution of creating a join in each query and put more effort in optimizing the query execution (mostly by tweaking my indexes).
I think that's really the best that one can do with Entity Framework. In the end, if the performance gets really bad, I'll probably go with John Wooley's suggestion:
This may be a situation where dropping to native ADO against a stored proc returning multiple results and using an internal temp table might be your best option for this operation. Use EF for the other 90% of your app.
Thanks to everyone who commented on this post...I appreciate everyone's input!
If the size of TableName is not too big to load the whole table you use
var tableNameById = DbContext.TableName.ToDictionary(x => x.ID);
to fetch the whole table and automatically put it in a local Dictionary with ID as key.
Another way is to just "force" the LINQ evaluation with .ToList(), in the case fetch the whole table and do the Where part locally with Linq2Objects.
var query1Lookup = new Hashset<int>(query1IDs);
var query2 = DbContext.TableName.ToList().Where(x => query1IDs.Contains(x.ID));
Edit:
Storing a list of ID:s from one query in a list and use that list as filter in another query can usually be rewritten as a join.
When I had similar problems with a heavy query that I wanted to reuse in several other queries I always went back to the solution of creating a join in each query and put more effort in optimizing the query execution (mostly by tweaking my indexes).
Since you are running a subsequent query off the results, take your first query and use it as a View on your SQL Server, add the view to your context, and build your LINQ queries against the view.
Have you considered composing your query as per this article (using the decorator design pattern):
Composed LINQ Queries using the Decorator Pattern
The premise is that, instead of enumerating your first (very constly) query, you basically use the decorator pattern to produce a chain of IQueryable that is a result of query 1 and query N. This way you always execute the filtered form of the query.
Hope this might help
Scenario: I have a table of User Profiles and a table of Colleagues. The colleagues table has a record for ever other user that a User Profile is following. In this query, I am grabbing all of the people who are following the current logged in user (getting their Record IDs from the Colleagues table and joining the User Profile table to get their details). Then, I am joining the Colleagues table again to see if the logged in user is following that user back.
Issue: It is my understanding that the best way to do a LEFT JOIN (in looking for the colleague record where the user is following the colleague back) is to add "DefaultIfEmpty()" to that join. When I add .DefaultIFEmpty() to that join, I get the following error message:
System.NotSupportedException: LINQ to Entities does not recognize the method 'System.Collections.Generic.IEnumerable'
If I remove the ".DefaultIFEmpty()" from that join, it works. However, it runs it as a regular JOIN, leaving out records where the user is not following the colleague back.
Code: Here is the code I am using:
var results = (from a1 in db.Colleague
join b1 in db.UserProfile on new {ColleagueId = a1.ColleagueId} equals
new {ColleagueId = b1.RecordId}
join d1 in db.UserProfile on new {RecordId = a1.OwnerId} equals
new {RecordId = d1.RecordId}
join c1 in db.Colleague
on new {OwnerId = b1.RecordId, Ignored = false, ColleagueId = a1.OwnerId}
equals new {c1.OwnerId, c1.Ignored, c1.ColleagueId} into c1Join
from c1 in c1Join.DefaultIfEmpty() // This is the .DefaultIfEmpty() breaking the query
where
b1.AccountName == userName &&
a1.Ignored == false
orderby
b1.LastName
select new
{
RecordId = (System.Int64?) d1.RecordId,
d1.AccountName,
d1.PreferredName,
d1.FirstName,
d1.LastName,
d1.PictureUrl,
d1.PublicUrl,
IsFollowing = c1.OwnerId < 1 ? 0 : 1
});
foreach (var result in results) // This is what throws the error
{
// Do stuff
}
Any ideas?
The SQL Provider for Entity Framework in version 3.5 of the .NET framework does not support DefaultIfEmpty(). Sorry, but could not find a better reference than this article: http://smehrozalam.wordpress.com/2009/06/10/c-left-outer-joins-with-linq/
You might want to try straight LINQ-to-SQL rather than ADO.NET Entity Framework. I believe that it works in LINQ-to-SQL. I've verified in the past that left joins work in 3.5 via LinqPad.
DefaultIfEmpty is supported only in EFv4+. First version of EF doesn't support DefaultInEmpty.
It looks like you need to create a default value to pass into DefaultIfEmpty(), since DefaultIsEmpty() takes a parameter.
DefaultIfEmpty() on MSDN