This question already has answers here:
Linq query or Lambda expression?
(5 answers)
Closed 7 years ago.
My project currently uses for following syntax for "linq"
var le = domainModel.EntityDataContext.Table_name_here.Query()
.WithFullReadCheck(domainModel.Security)
.Where(x => x.Metadata.AdvisorID == advisorId).FirstOrDefault();
It is said the above code is linq, being unaware about linq I decided to learn it. However, on https://msdn.microsoft.com/en-us/library/gg509017.aspx , things are pretty different.
what is being used in my code? Is it a version of linq? Is it something else?
What you are using is LINQ. There are 2 different notations you can use for writing LINQ - Lambda Syntax and Query Syntax
The difference is explained here: LINQ: Dot Notation vs Query Expression
There is more information on this on MSDN here: https://msdn.microsoft.com/en-us/library/bb308959.aspx
The MSDN articles starts explaining LINQ with SQL-like syntax ("query syntax") and then goes on to explain Lambda Expressions and Expression Trees ("lamba syntax").
That is extension method syntax for a query. The .Query().WithFullReadCheck() are much less common, but the rest .Where and .FirstOrDefault are pretty common query extensions. LINQ is a special syntax for expressing the same thing that can be done with chaining the query extension methods. Behind the scenes, they are identical.
See this question which gives an example of both LINQ and extension method syntax and has a good discussion of the differences between them: Extension methods syntax vs query syntax
Linq (language integrated query) is the language you use to query the data. You don't care where do they come from, that's the whole point of using it. You can use the same code to query arrays, collections, databases, xml files, directories, whatever... Linq translates your code in the background, so if you use it for instance to get the data from a database, it produces SQL to be sent to the database.
There are two syntax versions available:
1.) lambda syntax
2.) chainable methods
It doesn't matter which one you chose, just try to be consistent with it and use what makes you more comfortable / makes more sense in your situation.
LINQ is 'Language INtegrated Query'. In practical terms it means two things to most people:
The way that inline delegates are converted by the compiler into expression trees, not anonymous methods. That means that x => x.Metadata.AdvisorID == advisorId is not compiled into IL at compile time: instead it's compiled into code which builds an equivalent Expression object, which can be passed to a provider such as Entity Framework or LINQ to SQL, in order to generate a database query.
The other part it means to most people is the so-called "syntactic sugar", which effectively calls .Where(), .OrderBy() etc on your behalf:
from a in something
where a.Name == "Bob"
select a;
In both cases, the Queryable class in .NET provides extension methods for building a chain of Expression objects, for example .Where(), .OrderBy(), .Select() etc - these take an IQueryable (either untyped or with a generic parameter) and return another. They lightly wrap an Expression object at each point which represents the query as a whole.
Meaning:
someQueryable.Where(x => x.Id > 3).OrderBy(x => x.Date).Select(x => x.Ref)
Returns an object implementing IQueryable which holds an Expression which looks like:
Select(OrderBy(Where(someQueryable, x => x.Id > 3), x => x.Date), x => x.Ref)
... which is readable by LINQ providers to produce something like:
SELECT Ref FROM someTable WHERE Id > 3 ORDER BY Date
Finally, note that .Where(), .OrderBy() and the like are NOT limited to queryable/LINQ objects. They exist for IEnumerable (and IEnumerable<>), too - but this is not LINQ, and simply performs the operations at the instant that the method is called.
Related
What are the possible ways to create a LINQ expression dynamically, but using the query syntax? Is the query syntax a C# thing only, and if so, is the only viable way of creating such expressions using Roslyn dynamic compilation?
When writing LINQ expressions manually, I find them more natural when written using method chaining syntax, for example ctx.Foo.Where(foo => foo.Type.Name == "Bar") but there are some cases where I would need to write them like this:
from foo in ctx.Foo
join fooType in ctx.Types on foo.TypeId equals fooType.Id
where fooType.Name == "Bar"
I love how expression trees ensure type safety when creating expressions dynamically, but how would one create expressions using the query syntax?
Thanks everyone for your comments.
So it turns out it's not possible to do this because query syntax is just a C# language syntactic sugar.
Additionally, if someone else stumbles upon this question, take a look at the excellent answer by #Gert: https://stackoverflow.com/a/15599143/828023
That answer explains that the query syntax is "sugar", while the method syntax shows what really goes on under the hood, where for example join x in y on z equals x.something into somethingElse is actually a GroupJoin method call and there is no way to express this with expression trees without actually calling GroupJoin.
Is it correct to call this code snippet LINQ (Language Integrated Query)?
var lstMyStrings = new List<string>();
lstMyStrings.Where(aX => string.IsNullOrEmpty(aX))
.Skip(3)
.ToList();
I am confused because System.Linq is mandatory for this code.
However, when I see questions and answer's like this: .NET LINQ query syntax vs method chain
, then they're talking explicit about a method chain and not LINQ.
LINQ can be written in two different ways.
One is by writing a query using LINQ declarative query syntax:
var query = from x in source
where condition
select x.Property
And the other is by using LINQ's extension methods:
var query = source.Where(condition).Select(x => x.Property);
Both queries are identical and will produce the same result (well, compiler error in this over-simplified example but it's the thought that counts :-))
The c# compiler translates the query into method calls.
This means that everything you write as a query can be also written using method chains. Please note, however, that the opposite is false - Some queries can only be written using Linq's extension methods.
For further reading, here's what Microsoft have to say about it.
Note the second paragraph starts with this:
Query syntax and method syntax are semantically identical, but many people find query syntax simpler and easier to read.
btw, if it was'nt already clear, the reason that System.Linq is mandatory for the method chaining syntax also is because the linq extension methods belongs to this namespace.
I'm using entity framework to handle the database in a MVC application: I created a search engine for the application using several fields from start form, and setting privileges that comes from the user role, so the routine that create the select statement is growing, so sometimes I have this situation.
before i make a selection:
var orders = db.Orders.Where( ord => ord.Channel == 1 || ord.Channel == 2);
after in the source code, in some cases, type of users, filter combinations, and so on, I have to change this filter or remove this condition
orders = db.Orders.Where( ord => ord.Channel == 1 );
this is just a simple example, because doing this way i loose the other filters I have done in the Linq expression tree, and cause i have request to add features very often, it's very difficult to reorganize all the code to test all the prerequisite before to add conditions to the expression tree so i would like to know if is possible to remove a statement in the linq expression tree after has been added and before the tree is parsed and converted in a sql query
luca
Assuming I understand your question correctly, I believe the simplest possible approach is to defer the creation of the query until you have collected all the data needed to know what the query should look like, i.e. in this case it would mean you build the query only after you already know whether the condition actually needs to be applied.
If it is not possible for you to defer the creation of the query (it seems to me that is what you are saying), but you can still anticipate which conditions may need to be removed at a later point, I would consider introducing a Boolean sentinel in the query, e.g. based on your query snippet:
var orders = db.Orders.Where( ord =>
(isFirstConditionRelevant && ord.Channel == 1)
|| (isSecondConditionRelvant && ord.Channel == 2));
The variables 'isFirstConditionRelevant' and 'isSecondConditionRelevant' would be initially be set to true and would be captured by the query expression, but you could set to false at a later point (e.g. just before executing the query) if you needed the corresponding condition to have no effect. Notice that with this approach the SQL translation of the query will still contain the condition, but it will also contain a parameter for each sentinel that will short circuit the Boolean logic when the query is executed by the server.
Note also that any constants in your conditions (e.g. the '1' in 'ord.Channel == 1') will be translated to constant in the SQL. I would suggest using variables as this will introduce parameters in the SQL query that could increase the chances that the query plan can be reused on the server.
Another approach that could be useful is to take advantage of support for references to expression variables in the query, e.g. you could pass a variable of type Expression> to the Where clause and at a later point replace the value of the variable to the right predicate. If I remember correctly, LINQ to Entities supports using expression variables as predicate conditions over nested collections by applying the AsQueryable operator to the collection in the query. Otherwise you can take advantage of the support for AsExpandable in provided by the LINQ kit. You will find more information about this, as well as examples of using references to expression variables in the LINQ Kit home page: http://www.albahari.com/nutshell/linqkit.aspx.
The fourth and most complicated approach I would consider is to use my own visitor (e.g. derived from System.Linq.Expressions.ExpressionVisitor) to rewrite the LINQ expression tree (in this case to remove the condition from the predicate) before the query is executed. I don't have an expression visitor that does exactly that handy. Instead, I can provide a few pointers to articles that solve different parts of the problem:
This thread in StackOverflow describes how to use an expression visitor to do some query rewriting: Using a LINQ ExpressionVisitor to replace primitive parameters with property references in a lambda expression.
This great blog post by Alex James describes how to write an intercepting query provider that will call your expression rewriting visitor just be fore the query is executed: http://blogs.msdn.com/b/alexj/archive/2010/03/01/tip-55-how-to-extend-an-iqueryable-by-wrapping-it.aspx.
The following article describes a library that can be used to rewrite expressions based on rules: http://www.codeproject.com/Articles/24454/Modifying-LINQ-Expressions-with-Rewrite-Rules.
Hope this helps!
The following linq2entities code would appear to call the IEnumerable version of FirstOrDefault extension, because PersonHistories is an ICollection. At runtime however, it actually calls the IQueryable version.
var test = db.Persons.Where(d => d.PersonKey == 4)
.Select(f => f.PersonHistories.FirstOrDefault());
The problem I am having is the custom query provider I am using does not perform this automatic conversion, and I get the error "...ICollection cannot be used for parameter of type IQueryable". Thus requires explicitly calling AsQueryable to work around this, but for complex queries it gets very redundant and not feeling very DRY:
db.Persons.Where(d => d.PersonKey == 4)
.Select(f => f.PersonHistories.AsQueryable().FirstOrDefault());
I have dug around in the framework reference source trying to find the Linq 2 Entities visitors/provider stuff, but have had no luck(perhaps is not part of any of the open reference sources). How do the base providers accomplish this implicit use of AsQueryable?
I do understand that these are translated to expression trees.
I do understand that the Enumerable.FirstOrDefault is replaced with Queryable.FirstOrDefault by the provider. That is the premise of the question.
How do the base providers accomplish this implicit use of AsQueryable?
They don't. Your code doesn't really execute FirstOrDefault() at all. It builds an expression tree which represents the call, but that isn't executed directly. The query provider sees that, works out that the f.PersonHistories is actually based on an entity which is in the database, and converts the query appropriately.
As far as I understand the only thing LINQ supports, which Scala currently doesn't with its collection library, is the integration with a SQL Database.
As far as I understand LINQ can "accumulate" various operations and can give "the whole" statement to the database when queried to process it there, preventing that a simple SELECT first copies the whole table into data structures of the VM.
If I'm wrong, I would be happy to be corrected.
If not, what is necessary to support the same in Scala?
Wouldn't it possible to write a library which implements the collection interface, but doesn't have any data structures backing it but a String which gets assembled with following collection into the required Database statement?
Or am I completely wrong with my observations?
As the author of ScalaQuery, I don't have much to add to Stilgar's explanation. The part of LINQ which is missing in Scala is indeed the expression trees. That is the reason why ScalaQuery performs all its computations on Column and Table types instead of the basic types of those entities.
You declare a table as a Table object with a projection (tuple) of its columns, e.g.:
class User extends Table[(Int, String)] {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = id ~ name
}
User.id and User.name are now of type Column[Int] and Column[String] respectively. All computations are performed in the Query monad (which is a more natural representation of database queries than the SQL statements that have to be created from it). Take the following query:
val q = for(u <- User if u.id < 5) yield u.name
After some implicit conversions and desugaring this translates to:
val q:Query[String] =
Query[User.type](User).filter(u => u.id < ConstColumn[Int](5)).map(u => u.name)
The filter and map methods do not have to inspect their arguments as expression trees in order to build the query, they just run them. As you can see from the types, what looks superficially like "u.id:Int < 5:Int" is actually "u.id:Column[Int] < u.id:Column[Int]". Running this expression results in a query AST like Operator.Relational("<", NamedColumn("user", "id"), ConstColumn(5)). Similarly, the "filter" and "map" methods of the Query monad do not actually perform filtering and mapping but instead build up an AST that describes these operations.
The QueryBuilder then uses this AST to construct the actual SQL statement for the database (with a DBMS-specific syntax).
An alternative approach has been taken by ScalaQL which uses a compiler plugin to work directly with expression trees, ensure that they only contain the language subset which is allowed in database queries, and construct the queries statically.
I should mention that Scala does have experimental support for expression trees. If you pass an anonymous function as an argument to a method expecting a parameter of type scala.reflect.Code[A], you get an AST.
scala> import scala.reflect.Code
import scala.reflect.Code
scala> def codeOf[A](code: Code[A]) = code
codeOf: [A](code:scala.reflect.Code[A])scala.reflect.Code[A]
scala> codeOf((x: Int) => x * x).tree
res8: scala.reflect.Tree=Function(List(LocalValue(NoSymbol,x,PrefixedType(ThisType(Class(scala)),Class(scala.Int)))),Apply(Select(Ident(LocalValue(NoSymbol,x,PrefixedType(ThisType(Class(scala)),Class(scala.Int)))),Method(scala.Int.$times,MethodType(List(LocalValue(NoSymbol,x$1,PrefixedType(ThisType(Class(scala)),Class(scala.Int)))),PrefixedType(ThisType(Class(scala)),Class(scala.Int))))),List(Ident(LocalValue(NoSymbol,x,PrefixedType(ThisType(Class(scala)),Class(scala.Int)))))))
This has been used in the bytecode generation library 'Mnemonics', which was presented by its author Johannes Rudolph at Scala Days 2010.
With LINQ the compiler checks to see if the lambda expression is compiled to IEnumerable or to IQueryable. The first works like Scala collections. The second compiles the expression to an expression tree (i.e. data structure). The power of LINQ is that the compiler itself can translate the lambdas to expression trees. You can write a library that builds expression trees with interface similar to what you have for collection but how are you goning to make the compiler build data structures (instead of JVM code) from lambdas?
That being said I am not sure what Scala provides in this respect. Maybe it is possible to build data structures out of lambdas in Scala but in any case I believe you need a similar feature in the compiler to build support for databases. Mind you that databases are not the only underlying data source that you can build providers for. There are numerous LINQ providers to stuff like Active Directory or the Ebay API for example.
Edit:
Why there cannot be just an API?
In order to make queries you do not only use the API methods (filter, Where, etc...) but you also use lambda expressions as arguments of these methods .Where(x => x > 3) (C# LINQ). The compilers translate the lambdas to bytecode. The API needs to build data structures (expression trees) so that you can translate the data structure to the underlying data source. Basically you need the compiler to do this for you.
Disclaimer 1:
Maybe (just maybe) there is some way to create proxy objects that execute the lambdas but overload the operators to produce data structures. This would result in slightly worse performance than the actual LINQ (runtime vs compile time). I am not sure if such a library is possible. Maybe the ScalaQuery library uses similar approach.
Disclaimer 2:
Maybe the Scala language actually can provide the lambdas as an inspectable objects so that you can retrieve the expression tree. This would make the lambda feature in Scala equivalent to the one in C#. Maybe the ScalaQuery library uses this hypothetical feature.
Edit 2:
I did a bit of digging. It seems like ScalaQuery uses the library approach and overloads a bunch of operators to produce the trees at runtime. I am not entirely sure about the details because I am not familiar with Scala terminology and have hard time reading the complex Scala code in the article:
http://szeiger.de/blog/2008/12/21/a-type-safe-database-query-dsl-for-scala/
Like every object which can be used in or returned from a query, a table is parameterized with the type of the values it represents. This is always a tuple of individual column types, in our case Integers and Strings (note the use of java.lang.Integer instead of Int; more about that later). In this respect SQuery (as I’ve named it for now) is closer to HaskellDB than to LINQ because Scala (like most languages) does not give you access to an expression’s AST at runtime. In LINQ you can write queries using the real types of the values and columns in your database and have the query expression’s AST translated to SQL at runtime. Without this option we have to use meta-objects like Table and Column to build our own AST from these.
Very cool library indeed. I hope in the future it gets the love it deserves and becomes real production ready tool.
You probably want something like http://scalaquery.org/. It does exactly what #Stilgar answer suggests, except it's only SQL.
check out http://squeryl.org/