I have 1 LINQ which used so much. I try create the method which return this LINQ like:
public static System.Linq.Expressions.Expression<Func<MyEntity, bool>> GetFilteredEntity() {
return x => true/*Some condition*/;
}
public static Func<MyEntity, bool> GetFilteredEntity() {
return x => true/*Some condition*/;
}
And use this like
db.MyEntities.Where(GetFilteredEntity());
is successfull, But! I need use it like
db.ParentEntities.Where(entity => entity.MyEntities.Where(GetFilteredEntity()));
This code compiled too, but every time when i use it, i got the Error:
System.InvalidOperationException: There is already an open DataReader associated with this Command which must be closed first.
,even:
db.ParentEntities.Where(entity => entity.MyEntities.Where(GetFilteredEntity())).ToList();
throw this Exception too.
But,
db.ParentEntities.Where(entity => entity.MyEntities.Where(x => true/*Some condition*/))
still works fine!
So why it happend, and have some way to get round this?
Final working code
public static Expression<Func<MyEntity, bool>> GetFilteredEntity() {
return x => true/*Some condition*/;
}
and
var expression = GetFilteredEntity();
db.ParentEntities.Where(entity => entity.MyEntities.AsQueryable().Where(expression ));
Also .AsQueryable() thanks to Passing func as parameter in Linq to Entities and 'Internal .NET Framework Data Provider error 1025' error
In your first example the function is called and translated into an expression before it is even sent to the query provider. In the next two examples the function call is embedded within an expression that is sent to the query provider, and that query provider doesn't know what to do with that function call, so it just throws an exception. When you embed the actual expression in another expression, there is no function call to confuse the query provider.
As for a solution, just pull out the function call into a variable. The query provider is smart enough to see that you used a closed over variable, and will pull out its value. For a function call it's just not sure if it should evaluate it or try to translate it into something that should be done on the DB's end. Trying to do some of both would just be very confusing and hard to work with, both for the query provider, and for people using it. To simplify matters, function calls with expressions are never executed prior to sending the query. As for a closed over variable, there is no other way that it could be treated, so there isn't any other behavior to confuse it with.
var expression = GetFilteredEntity();
db.ParentEntities.Where(entity => entity.MyEntities.Where(expression ));
Looks like LazyLoading might be the culprit, have you tried popping the ToList() on the parameters?
db.ParentEntities.Where(entity => entity.MyEntities.Where(GetFilteredEntity()).ToList());
Related
I'm trying to run a LINQ query against an Azure DocumentDB collection. When I run my query I keep getting an AggregateException that contains an InvalidOperationException with the message:
Nullable object must have a value
I've reduced this issue to the following (somewhat contrived) example:
When I run this code, I get the above mentioned exception thrown from the call to ToArray()
public class MyDocument { ... }
public void RunQuery()
{
var query = documentDbClient
.CreateDocumentQuery<MyDocument>()
.Where(doc => GetDoc(doc) != null);
var results = query.ToArray()
}
public MyDocument GetDoc(MyDocument myDocument)
{
return myDocument;
}
In contrast, when I run the code below no exception is thrown and I get back good results from the DocumentDB collection.
public void RunQuery()
{
var query = documentDbClient
.CreateDocumentQuery<MyDocument>()
.Where(doc => doc != null);
var results = query.ToArray()
}
Why the difference in behavior between the two code samples?
Notes:
- While GetDoc() is a stand in for my more complex predicate logic, the code above reproduces the issue exactly. I'm not withholding any shenanigans inside GetDoc() or other methods :)
- The issue occurs even when GetDoc() is made static.
- Just tried to reproduce with a List<MyDocument> instead of documentDbClient and no exception was thrown. Indicates something in the underlying data provider = Azure DocumentDB's IDatabaseClient.
With help from #andrew-liu and #will (thanks!) I've been able to figure out that:
At the time of the call to Where() in my above example, the predicate inside is interpreted (rather than executed) as a LINQ expression. At the time of the call to ToArray() the LINQ expression is executed against the DocumentDB .NET SDK's LINQ provider.
Because the SDK is not aware of my GetDoc() method, it throws the exception with the cryptic message "Nullable object must have a value".
In v1.9 of the DocumentDB SDK for .Net the error message is much clearer. A DocumentQueryException is thrown with a message like "Method "GetDoc()" is not supported".
You can get similar feedback from v1.8 of the SDK by invoking ToString() on the query like so: query.ToString().
I developed the extension method:
public static IQueryable<T> Get<T>(this DataContext dataContext) where T : class
{
return dataContext.GetTable<T>();
}
Then I tried to use it:
dataContext.Get<XDbCurrency>().
Where(c => c.Id == XCurrencyId.USD).
SelectMany(
c =>
dataContext.Get<XDbCurrencyRate>().
Where(cr =>
cr.CurrencyId == c.Id &&
cr.StartOn <= DateTime.UtcNow &&
cr.EndOn > DateTime.UtcNow).
Take(1),
(c, cr) => cr).
ToArray();
I get the exception Member access 'XCurrencyId CurrencyId' of 'X.XDbCurrencyRate' not legal on type 'System.Data.Linq.Table``1[X.XDbCurrencyRate]. When I change Get back to GetTable - everything works! When I change the return type of Get from IQueryable<T> to Table<T> - it crashes.
UPDATE. I created XDataContextBase class, moved Get method there, inherited DBML-generated XDataContext from XDataContextBase - unfortunately it does not help. Magic!!
UPDATE. this.GetTable<XDbCurrencyRate> works, this.XDbCurrencyRates works, this.Get<XDbCurrencyRate> not works either Get is extension method or method in base class. By the way, internally this.XDbCurrencyRates is implemented absolutely like my Get. Magic!!
UPDATE. Looks like Linq-to-SQL supports only two things: either direct call of DataContext.GetTable<> or call of any-named property that returns System.Data.Linq.Table<XDbCurrencyRate>. If I create System.Data.Linq.Table<XDbCurrencyRate> Foo - it works, but when I create IQueryable<XDbCurrencyRate> Foo - it crashes with Queries with local collections are not supported.. Magic!!
Any ideas?
This is because the initial Get call is not part of the query tree. Just a normal method call. The second call is embedded in the tree and LINQ to SQL has no idea what that method means.
I know that EF would give up here immediately. I believe L2S is basically capable of running such a method locally and trying to inline the query that it returns. Here, that seems to now work out (or I'm wrong about that).
What I did was to create a custom LINQ provider that rewrites expression trees to execute such methods locally, inline their results into the query tree and then forward to LINQ to SQL. This is not trivial.
Maybe you can make AsExpandable from the so called "LINQ kit" (approximate spelling) work.
The following method works properly in my service layer:
public override IList<City> GetAll()
{
var query = from item in _tEntities
select item;
query.Load();
return _tEntities.Local;
}
but when i try to run following method, the method returns old data loaded by GetAll() method.
public override IList<City> GetAll(Func<City, bool> predicate)
{
var query = from item in _tEntities
select item;
query.Where<City>(predicate);
query.Load();
return _tEntities.Local;
}
What is the problem and how can i fix them?
How can i use local method in this sample and reload new data into local(cache)?
You are looking at the wrong problem. What you are most likely seeing is a result of the fact that when you do the first query, the local cache is empty. So it only returns the results from your query. But when you do the second, it's returning the results of your first query AND your second query.
This comes down to the fact that you are using a shared DbContext between all your methods. Local contains a cache of all records the context has retrieved, not just the most recent query.
The correct solution is to not use Local in this manner. Even better, don't use a shared context since this can lead to context cache bloat.
I'm not too sure what you are trying to achieve with a .Load method here but it seems like you want the following.
public override IList<City> GetAll(Func<City, bool> predicate)
{
return _tEntities.Where<City>(predicate).ToList();
}
query.Where<City>(predicate);
This doesn't change query. The query.Load() on the next line ignores the predicate: you're calling query.Load() and not query.Where<City>(predicate).Load(). It's as if you had written
int i = 3;
i + 1;
Console.WriteLine(i); // still prints 3
In that example, C# does not really actually allow an addition to be used as a statement, but .Where(predicate) is a method call, and method calls can be used as such, even if they return values.
This is not your only issue (see the other answers), but my guess is that this issue is the one that leads to the unexpected results you're seeing.
I have written a custom IQueryProvider class that takes an expression and analyses it against a SQL database (I know I could use Linq2Sql but there are some modifications and tweaks that I need that unfortunately make Linq2Sql unsuitable). The class will identify and do something with the properties that are marked (using attributes) but any that aren't I would like to be able to pass the expression on to a LinqToObject provider and allow it to filter the results after.
For example, suppose I have the following linq expression:
var parents=Context.Parents
.Where(parent=>parent.Name.Contains("T") && parent.Age>18);
The Parents class is a custom class that implements IQueryProvider and IQueryable interfaces, but only the Age property is marked for retrieval, so the Age property will be processed, but the Name property is ignored because it is not marked. After I've finished processing the Age property, I'd like to pass the whole expression to LinqToObjects to process and filter, but I don't know how.
N.B. It doesn't need to remove the Age clause of the expression because the result will be the same even after I've processed it so I will always be able to send the whole expression on to LinqToObjects.
I've tried the following code but it doesn't seem to work:
IEnumerator IEnumerable.GetEnumerator() {
if(this.expression != null && !this.isEnumerating) {
this.isEnumerating = true;
var queryable=this.ToList().AsQueryable();
var query = queryable.Provider.CreateQuery(this.expression);
return query.GetEnumerator();
}
return this;
}
this.isEnumerating is just a boolean flag set to prevent recursion.
this.expression contains the following:
{value(namespace.Parents`1[namespace.Child]).Where(parent => ((parent.Name.EndsWith("T") AndAlso parent.Name.StartsWith("M")) AndAlso (parent.Test > 0)))}
When I step through the code, despite converting the results to a list, it still uses my custom class for the query. So I figured that because the class Parent was at the beginning of the expression, it was still routing the query back to my provider, so I tried setting this.expression to Argument[1] of the method call so it was as such:
{parent => ((parent.Name.EndsWith("T") AndAlso parent.Name.StartsWith("M")) AndAlso (parent.Test > 0))}
Which to me looks more like it, however, whenever I pass this into the CreateQuery function, I get this error 'Argument expression is not valid'.
The node type of the expression is now 'Quote' though and not 'Call' and the method is null. I suspect that I just need to make this expression a call expression somehow and it will work, but I'm not sure how to.
Please bear in mind that this expression is a where clause, but it may be any kind of expression and I'd prefer not to be trying to analyse the expression to see what type it is before passing it in to the List query provider.
Perhaps there is a way of stripping off or replacing the Parent class of the original expression with the list provider class but still leaving it in a state that can just be passed in as expression into the List provider regardless of the type of expression?
Any help on this would be greatly appreciated!
You were so close!
My goal was to avoid having to "replicate" the full mind-numbingly convoluted SQL-to-Object expressions feature set. And you put me on the right track (thanks!) here's how to piggy-back SQL-to-Object in a custom IQueryable:
public IEnumerator<T> GetEnumerator() {
// For my case (a custom object-oriented database engine) I still
// have an IQueryProvider which builds a "subset" of objects each populated
// with only "required" fields, as extracted from the expression. IDs,
// dates, particular strings, what have you. This is "cheap" because it
// has an indexing system as well.
var en = ((IEnumerable<T>)this.provider.Execute(this.expression));
// Copy your internal objects into a list.
var ar = new List<T>(en);
var queryable = ar.AsQueryable<T>();
// This is where we went wrong:
// queryable.Provider.CreateQuery(this.expression);
// We can't re-reference the original expression because it will loop
// right back on our custom IQueryable<>. Instead, swap out the first
// argument with the List's queryable:
var mc = (MethodCallExpression)this.expression;
var exp = Expression.Call(mc.Method,
Expression.Constant(queryable),
mc.Arguments[1]);
// Now the CLR can do all of the heavy lifting
var query = queryable.Provider.CreateQuery<T>(exp);
return query.GetEnumerator();
}
Can't believe this took me 3 days to figure out how to avoid reinventing wheel on LINQ-to-Object queries.
Why does this throw System.NotSupportedException?
string foo(string f) { return f; }
string bar = "";
var item = (from f in myEntities.Beer
where f.BeerName == foo(bar)
select f).FirstOrDefault();
Edit: Here's an MSDN reference that (kind of) explains things...
Any method calls in a LINQ to Entities
query that are not explicitly mapped
to a canonical function will result in
a runtime NotSupportedException
exception being thrown. For a list of
CLR methods that are mapped to
canonical functions, see CLR Method to
Canonical Function Mapping.
See also http://mosesofegypt.net/post/LINQ-to-Entities-what-is-not-supported.aspx
EDIT: Okay, the code blows up because it doesn't know what to do with the call to foo(). The query is built up as an expression tree which is then converted to SQL.
The expression tree translator knows about various things - such as string equality, and various other methods (e.g. string.StartsWith) but it doesn't know what your foo method does - foo() is a black box as far as it's concerned. It therefore can't translate it into SQL.
The second version will fail as soon as you try to iterate over it. You can't use a locally defined method in an IQueryable<> where clause (of course, you can, but it will fail when the LINQ provider tries to translate it into SQL).
Because in the 2nd query no actual query is executed. Try adding ToList() where SingleOrDefault() is.
It's probably because the SQL-generation functionality isn't able to determine what to do with your foo() function, so can't generate output for it.