i am trying to figure out why nhibernate query iterates on values instead of using joins internally. due to this iteration it becomes slower, cause it will iterates all values one by one.
i.e. it generates n no. of queries and execute it one by one instead of joins.
documentClrType is evaluated dynamically at runtime, so i can't use directly in QueryOver<> syntax
documentClrType is FactSheetPrivate as of now.
i observed query in logger it is created something like this.
select * from foo where col1=#val1
select * from foo where col1=#val2
select * from foo where col1=#val3
select * from foo where col1=#val4
so,
how could i turn this query to joins instead of iterations?
what could be syntax for this dynamic types with query over?
i am bit new to nhibernate, any guidance will be appreciated.
var criteria=
store.Session.
CreateCriteria(documentClrType)
.Add(Restrictions.Disjunction()
.Add(Restrictions.Le("CreationDate",)DateTime.Now )
.Add(Restrictions.Le("AccurateDate",)DateTime.Now )
)
criteria=criteria.CreateCriteria("Entity")
.Add(Restrictions.Eq("DBTypeString",receiverType))
return criteria.List<IDocument>()
// at this lines instead of join query iterates value one by one
following screen display entity nhibernate xml file
following screen display packets nhibernate XML file
NHibernate has a feature to batch the selects
configuration.SetProperty(NHibernate.Cfg.Environment.DefaultBatchFetchSize, "20")
or use eager fetching
return criteria.SetFetchMode("TheReferenceOrCollectionProperty", FetchMode.Eager).List<IDocument>()
Related
I have a Linq query that reads from a SQL table and 1 of the fields it returns are from a custom function (in C#).
Something like:
var q = from my in MyTable
select new
{
ID = my.ID,
Amount = GetAmount(ID)
};
If I do a q.Dump() in LinqPad, it shows the results, which tells me that it runs the custom function without trying to send it to SQL.
Now I want to union this to another query, with:
var q1 = (from p in AnotherQuery.Union(q)...
and the I get the error that Method has no supported translation to SQL.
So, my logic tells me that I need to dump q in memory and then try to union to that. I've tried doing that with ToList() and creating a secondary query that populates itself from the List, but that leads to a long list of different errors. Am I on the right track, by trying to get q in memory and union on that, or are there better ways of doing this?
You can't use any custom functions in a LINQ query that gets translated - only the functions supported by the given LINQ provider. If you want your query to happen on the server, you need to stick with the supported functions (even if it sometimes means having to inline code that would otherwise be reused).
The difference between your two queries boils down to when (and where) the projection happens. In your first case, the data from MyTable is returned from the DB - in your sample, just the ID. Then, the projection happens on top of this - the GetAmount method is called in your application for each of ID.
On the other hand, there's no such way for this to happen in your second query, since you're not using GetAmount in the final projection.
You either need to replace the custom function with inlined query the provider understands, or refactor all your queries to use the supported functions in addition with whatever you need to do in-memory. There's no point in giving you any sample code, since it depends entirely on your actual query, and what you're really trying to query for.
I have an app using Entity Framework. I want to add a tree view listing products, grouped by their categories. I have an old SQL query that will grab all of the products and categories and arrange them into parent nodes and children. I am trying to translate it into LINQ that uses the EF. But the SQL has a WITH sub-query that I am not familiar with using. I have tried using Linqer and LinqPad to sort it out, but they choke on the WITH clause and I am not sure how to fix it. Is this sort of thing possible in LINQ?
Here is the query:
declare #id int
set #id=0
WITH ChildIDs(id,parentid,type,ChildLevel) AS
(
SELECT id,parentid,type,0 AS ChildLevel
FROM dbo.brooks_product
WHERE id = #id
UNION ALL
SELECT e.id,e.parentid,e.type,ChildLevel + 1
FROM dbo.brooks_product AS e
INNER JOIN ChildIDs AS d
ON e.parentid = d.id
WHERE showitem='yes' AND tribflag=1
)
SELECT ID,parentid,type,ChildLevel
FROM ChildIDs
WHERE type in('product','productchild','productgroup','menu')
ORDER BY ChildLevel, type
OPTION (MAXRECURSION 10);
When I run the query, I get data that looks like this (a few thousand rows, truncated here):
ID.....parentid.....type.....ChildLevel
35429..0............menu.....1
49205..0............menu.....1
49206..49205........menu.....2
169999.49206........product..3
160531.169999.......productchild..4
and so on.
The WITH block is a Common Table Expression, and in this case is used to create a recursive query.
This will be VERY difficult in Linq as Linq doesn't play well with recursion. If you need all of the data on one result set that a Stored Procedure would be easier. Another option is to do the recursion in C# (not in Linq but a recursive function) and do multiple round-trips. The performance will not be as good but if you result set is small it may not make much difference (and you will get a better object model).
You may be able to solve this using LINQ to Entities, but it is non-trivial and I suspect it will be very time consuming.
In situations like this, you may prefer to build a SQL View or Table-Valued Function that returns the results for which you're looking. Then import that View or Table-Valued Function into your EF model and you can pull data directly from it using LINQ.
Querying the View in LINQ is no different than querying a table.
To get data from a Table-Valued Function in LINQ, you pass the function's parameters in after the name of the function, like so:
var query = from tvf in _db.MyTableValuedFunction(parameters)
select tvf;
EDIT
As suggested by #thepirat000, Table-Valued Function support is not available in Entity Framework versions prior to version 5. In order to use this functionality, EF must be running with .NET 4.5 or higher.
At the end of the day, I could not get this to work. I ended up writing out a SQL query dynamically and sending that straight to the database. It works fine, and I am not relying on any direct user input so there is no chance of SQL injection. But it seems so old school! For the rest of my program I am using EF and LINQ.
Thanks for the replies!
I am trying to implement some search functionality within our app and have a situation where a User can select multiple Topics from a list and we want to return all activities that match at least one of the selected Topics. Each Activity can have 0-to-many Topics.
I can write a straight SQL query that gives me the results I want like so:
SELECT *
FROM dbo.ACTIVITY_VERSION av
WHERE ACTIVITY_VERSION_ID IN (
SELECT ACTIVITY_VERSION_ID
FROM dbo.ACTIVITY_TOPIC at
WHERE at.TOPIC_ID IN (3,4)
)
What I can't figure out is how to write a LINQ query (we are using Linq to Sql) that returns the same results.
I've tried:
activities.Where(x => criteria.TopicIds.Intersect(x.TopicIds).Any());
this works if activities is a list of in memory objects (i.e. a Linq to Objects query), but I get an error if I try to use the same code in a query that hits the database. The error I receive is:
Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
I believe that this means that Linq to Sql doesn't know how to translate either Intersect or Any (or possibly both). If that is the case, I understand why it isn't working, but I still don't know how to make it do what I want it to and my Google-fu has not provided me with anything that works.
Haven't tested it. But this is how you ll go about it.
List<int> IDs = new List<int>();
IDs.Add(3);
IDs.Add(4);
var ACTIVITY_VERSION_IDs = ACTIVITY_TOPIC
.Where(AT => IDs.Contains(AT.TOPIC_ID))
.Select(AT=>AT.ACTIVITY_VERSION_ID)
var results = ACTIVITY_VERSION
.Where(AV => ACTIVITY_VERSION_IDs.Contains(AV.ACTIVITY_VERSION_ID))
I have a situation where my application constructs a dynamic LINQ query using PredicateBuilder based on user-specified filter criteria (aside: check out this link for the best EF PredicateBuilder implementation). The problem is that this query usually takes a long time to run and I need the results of this query to perform other queries (i.e., joining the results with other tables). If I were writing T-SQL, I'd put the results of the first query into a temporary table or a table variable and then write my other queries around that. I thought of getting a list of IDs (e.g., List<Int32> query1IDs) from the first query and then doing something like this:
var query2 = DbContext.TableName.Where(x => query1IDs.Contains(x.ID))
This will work in theory; however, the number of IDs in query1IDs can be in the hundreds or thousands (and the LINQ expression x => query1IDs.Contains(x.ID) gets translated into a T-SQL "IN" statement, which is bad for obvious reasons) and the number of rows in TableName is in the millions. Does anyone have any suggestions as to the best way to deal with this kind of situation?
Edit 1: Additional clarification as to what I'm doing.
Okay, I'm constructing my first query (query1) which just contains the IDs that I'm interested in. Basically, I'm going to use query1 to "filter" other tables. Note: I am not using a ToList() at the end of the LINQ statement---the query is not executed at this time and no results are sent to the client:
var query1 = DbContext.TableName1.Where(ComplexFilterLogic).Select(x => x.ID)
Then I take query1 and use it to filter another table (TableName2). I now put ToList() at the end of this statement because I want to execute it and bring the results to the client:
var query2 = (from a in DbContext.TableName2 join b in query1 on a.ID equals b.ID select new { b.Column1, b.column2, b.column3,...,b.columnM }).ToList();
Then I take query1 and re-use it to filter yet another table (TableName3), execute it and bring the results to the client:
var query3 = (from a in DbContext.TableName3 join b in query1 on a.ID equals b.ID select new { b.Column1, b.column2, b.column3,...,b.columnM }).ToList();
I can keep doing this for as many queries as I like:
var queryN = (from a in DbContext.TableNameN join b in query1 on a.ID equals b.ID select new { b.Column1, b.column2, b.column3,...,b.columnM }).ToList();
The Problem: query1 is takes a long time to execute. When I execute query2, query3...queryN, query1 is being executed (N-1) times...this is not a very efficient way of doing things (especially since query1 isn't changing). As I said before, if I were writing T-SQL, I would put the result of query1 into a temporary table and then use that table in the subsequent queries.
Edit 2:
I'm going to give the credit for answering this question to Albin Sunnanbo for his comment:
When I had similar problems with a heavy query that I wanted to reuse in several other queries I always went back to the solution of creating a join in each query and put more effort in optimizing the query execution (mostly by tweaking my indexes).
I think that's really the best that one can do with Entity Framework. In the end, if the performance gets really bad, I'll probably go with John Wooley's suggestion:
This may be a situation where dropping to native ADO against a stored proc returning multiple results and using an internal temp table might be your best option for this operation. Use EF for the other 90% of your app.
Thanks to everyone who commented on this post...I appreciate everyone's input!
If the size of TableName is not too big to load the whole table you use
var tableNameById = DbContext.TableName.ToDictionary(x => x.ID);
to fetch the whole table and automatically put it in a local Dictionary with ID as key.
Another way is to just "force" the LINQ evaluation with .ToList(), in the case fetch the whole table and do the Where part locally with Linq2Objects.
var query1Lookup = new Hashset<int>(query1IDs);
var query2 = DbContext.TableName.ToList().Where(x => query1IDs.Contains(x.ID));
Edit:
Storing a list of ID:s from one query in a list and use that list as filter in another query can usually be rewritten as a join.
When I had similar problems with a heavy query that I wanted to reuse in several other queries I always went back to the solution of creating a join in each query and put more effort in optimizing the query execution (mostly by tweaking my indexes).
Since you are running a subsequent query off the results, take your first query and use it as a View on your SQL Server, add the view to your context, and build your LINQ queries against the view.
Have you considered composing your query as per this article (using the decorator design pattern):
Composed LINQ Queries using the Decorator Pattern
The premise is that, instead of enumerating your first (very constly) query, you basically use the decorator pattern to produce a chain of IQueryable that is a result of query 1 and query N. This way you always execute the filtered form of the query.
Hope this might help
I've been searching here and Google, but I'm at a loss. I need to let users search a database for reports using a form. If a field on the form has a value, the app will get any reports with that field set to that value. If a field on a form is left blank, the app will ignore it. How can I do this? Ideally, I'd like to just write Where clauses as Strings and add together those that are not empty.
.Where("Id=1")
I've heard this is supposed to work, but I keep getting an error: "could not be resolved in the current scope of context Make sure all referenced variables are in scope...".
Another approach is to pull all the reports then filter it one where clause at a time. I'm hesitant to do this because 1. that's a huge chunk of data over the network and 2. that's a lot of processing on the user side. I'd like to take advantage of the server's processing capabilities. I've heard that it won't query until it's actually requested. So doing something like this
var qry = ctx.Reports
.Select(r => r);
does not actually run the query until I do:
qry.First()
But if I start doing:
qry = qry.Where(r => r.Id = 1).Select(r => r);
qry = qry.Where(r => r.reportDate = '2010/02/02').Select(r => r);
Would that run the query? Since I'm adding a where clause to it. I'd like a simple solution...in the worst case I'd use the Query Builder things...but I'd rather avoid that (seems complex).
Any advice? :)
Linq delays record fetching until a record must be fetched.
That means stacking Where clauses is only adding AND/OR clauses to the query, but still not executing.
Execution of the generated query will be done in the precise moment you try to get a record (First, Any etc), a list of records(ToList()), or enumerate them (foreach).
.Take(N) is not considered fetching records - but adding a (SELECT TOP N / LIMIT N) to the query
No, this will not run the query, you can structure your query this way, and it is actually preferable if it helps readability. You are taking advantage of lazy evaluation in this case.
The query will only run if you enumerate results from it by using i.e. foreach or you force eager evaluation of the query results, i.e. using .ToList() or otherwise force evaluation, i.e evaluate to a single result using i.e First() or Single().
Try checking out this dynamic Linq dll that was released a few years back - it still works just fine and looks to be exactly what you are looking for.