As a fairly junior developer, I'm running into a problem that highlights my lack of experience and the holes in my knowledge. Please excuse me if the preamble here is too long.
I find myself on a project that involves my needing to learn a number of new (to me) technologies, including LINQ (to OBJECTS and to XML for purposes of this project) among others. Everything I've read to this point suggests that to utilize LINQ I'll need to fully understand the following (Delegates, Anonymous Methods and Lambda Expressions).
OK, so now comes the fun. I've CONSUMED delegates in the past as I have worked with the .NET event model, but the majority of the details have been hidden from me (thanks Microsoft!). I understand that on a basic level, delegate instances are pointers to methods (a gross over-simplification, I know).
I understand that an anonymous method is essentially an in-line unnamed method generally (if not exclusively) created as a target for a delegate.
I also understand that lambdas are used in varying ways to simplfy syntax and can be used to point a simple anonymous method to a delegate.
Pardon me if my any of my descriptions are WAY off here, this is the basic level to which I understand these topics.
So, the challenge:
Can anyone tell me if at least on a basic level if my understanding of these items is even close? I'm not looking for complex esoteric minutiae, just the basics (for now).
To what degree do I need to truly understand these concepts before applying LINQ in a project to reasonable effect? I want to understand it fully and am willing to spend the time. I simply may not HAVE the time to fully grok all of this stuff before I need to produce some work.
Can anyone point me to some good articles that explain these subjects and apply them to "real world" examples so that I can get my head around the basics of the topics and application of them? What I mean by real world, is how might I use this in the context of "Customers and Invoices" rather than abstract "Vectors and Shapes" or "Animals and Cows". The scenario can be somewhat contrived for demonstration purposes, but hopefully not strictly academic. I have found a number of examples on-line and in books, but few seem to be "Plain English" explanations.
Thank you all in advance for your patience, time and expertise.
Where can i find a good in depth guide to C# 3?
1) Your knowledge so far seems ok. Lambda expressions are turned into anonymous methods or System.Linq.Expressions.Expression's, depending on context. Since you aren't using a database technology, you don't need to understand expressions (all lambdas will be anonymous methods). You didn't list Extension methods, but those are very important (and easy) to understand. Make sure you see how to apply an extension method to an interface - as all the functionality in linq comes from System.Linq.Enumerable - a collection of extention methods against IEnumerable(Of T).
2) You don't need a deep understanding of lambdas.
The arrow syntax ( => ) was the biggest hurdle for me. The arrow separates the signature and the body of the lambda expression.
Always remember : Linq methods are not executed until enumerated.
Watch out for using loop variables in a lambda. This is a side effect from deferred execution that is particularly tricky to track down.
3) Sure, Here are some of my answers that show linq method calls - some with xml.
List splitting
Simple Xml existence search
Xml projection - shape change
1) Those descriptions sound pretty accurate to me. Sometimes anonymous methods and lambda expressions will need to create a new type to put the target of the delegate in, so they can act as closures.
2/3) I would read up a bit until you're happy with delegates, anonymous methods and lambda expressions. I dedicate a chapter to the delegate-related changes in each of C# 2.0 and C# 3.0 in C# in Depth, although of course other books go into detail too. I have an article as well, if that helps.
As for examples - delegates are used for many different purposes. They're all different ways of looking at the same functionality, but they can feel very different:
Providing the code to call when you start a new thread
Reacting to UI events
Providing the filter, selection, ordering etc for a LINQ query
Providing a callback for when an asynchronous operation has finished
If you have any specific situations you'd like an example of, that would be easier to answer.
EDIT: I should point out that it's good news that you're only working with LINQ to Objects and LINQ to XML at the moment, as that means you don't need to understand expression trees yet. (They're cool, but one step at a time...) LINQ to XML is really just an XML API which works nicely with LINQ - from what I remember, the only times you'll use delegates with LINQ to XML are when you're actually calling into LINQ to Objects. (That's very nice to do, admittedly - but it means you can reuse what you've already learned.)
As you've already got C# in Depth, chapters 10 and 11 provide quite a few examples of using lambda expressions (and query expressions which are translated into lambda expressions) in LINQ. Chapter 5 has a few different examples of delegate use.
Read this...
http://linqinaction.net/
..and all you're question will be answered!!!
Related
So, I have this new project. My company uses the SalesForce.com cloud to store information about day-to-day operations. My job is to write a new application that will, among other things, more seamlessly integrate the CRUD operations of this data with existing in-house application functionality.
The heart of the Salesforce WSDL API is a set of "query()" web methods that take the query command as a string. The syntax of the query is SQL-ish, but not quite (they call it SOQL). I'm not a fan of "magic strings", so I'd like to use Linq in the codebase, and parse the IQueryable into the SOQL query I need in the wrapper for the service. It's certainly possible (L2E, L2Sql), but I'd like to know if there's a shortcut, 'cause if I say it'll take more than a day or two to roll my own, I'll be "encouraged" to find another method (most likely a method for each general-use query, which was the method used in older apps). If I succeed in making a general-purpose SOQL parser, we can use it in several other upcoming apps, and I'll be a hero. Even if I make a simple one that only supports certain query structures, it'll go a long way by allowing me to proceed with the current project in a Linq-y way, and I can expand on it in my free time.
Here are the options as I see it:
Look harder for an existing Linq2SOQL provider (My Google-fu is failing me here, or else there simply isn't one; the only .NET wrapper only mentions Linq as a nice-to-have).
Build an expression tree parser. It needs to support, at least, the Select and Where method calls, and needs to either parse lambdas or manipulate their method bodies to get the operations and projections needed. This seems to be a rather massive task, but like I said, it's certainly possible.
Wrap the service in Linq2Sql or a similar existing Linq provider that will allow me to extract a close-enough query string, polish it up and pass it to the service. There must be dozens out there (though none that just drop in, AFAIK).
Call Expression.ToString() (or Expression.DebugView), and manipulate that string to create the SOQL query. It'll be brittle, it'll be ugly (behind the scenes), and it will support only what I'm explicitly looking for, but it will provide a rudimentary translation that will allow me to move on.
What do you guys think? Is building a Linq parser more than a two-day task for one guy? Would a bodged-up solution involving an existing Linq provider possibly do it? Would it be terrible to chop up the expression string and construct my query that way?
EDIT: Thanks to Kirk for the grounding. I took some more looks at what I'd be required to do for even a basic SOQL parser, and it's just beyond the scope of me getting working application code written on any feasible schedule. For instance, I have to build a select list from either the Select() method lambda or a default one from all known columns from my WSDL object, a task I hadn't even thought of (I was focusing more on the Where parsing). I'm sure there are many other "unknown unknowns" that could turn this into a pretty big deal. I found several links which shows the basics of writing a Linq provider, and though they all try to make it simple, it's just not going to be feasible time-wise right now. I'll structure my repository, for now, using named methods that encapsulate named queries (a constant class of formattable query strings should reduce the amount of head-scratching in maintenance). Not perfect, but far more feasible. If and when a Linq2SOQL provider gets off the ground, either in-house or open-source, we can refactor.
For others looking for Linq provider references, here are those helpful links I found:
Building a Linq Provider
Walkthrough: Creating an IQueryable LINQ Provider
Linq: Building an IQueryable provider series - in 17 parts! <-- this one, though long and involved, has a lot of real in-depth explanations and is good for "first-timers".
Let's take them one at a time:
Look harder for an existing Linq2SOQL provider (My Google-fu is failing me here, or else there simply isn't one; the only .NET wrapper only mentions Linq as a nice-to-have).
Yeah, I doubt one exists already, but hopefully you can find one.
Build an expression tree parser. It needs to support, at least, the Select and Where method calls, and needs to either parse lambdas or manipulate their method bodies to get the operations and projections needed. This seems to be a rather massive task, but like I said, it's certainly possible.
This is absolutely the way to go if you really are serious about this in the long-run.
Wrap the service in Linq2Sql or a similar existing Linq provider that will allow me to extract a close-enough query string, polish it up and pass it to the service. There must be dozens out there (though none that just drop in, AFAIK).
What do you mean by "drop in"? You can easily get the SQL straight from L2S.
Call Expression.ToString() (or Expression.DebugView), and manipulate that string to create the SOQL query. It'll be brittle, it'll be ugly (behind the scenes), and it will support only what I'm explicitly looking for, but it will provide a rudimentary translation that will allow me to move on.
I would strongly discourage you from this approach, as, at minimum, it will be at least as difficult as parsing the expression trees properly. If anything, in order to use this, you'd have to first put the parsed strings into a proper object model -- i.e. the existing expression trees you're starting with.
Really, you should think about building a query provider and doing this right. I think two days is a bit of a stretch though to get something working in even a primitive sense, though it might be possible. IMO, you should research it some at home and toy around with it so you have some familiarity with the basic pieces and parts. Then you might barely be able to get some usable queries going after two days.
Honestly though, fully implementing this kind of project is really in the realm of several weeks, if not months -- not days.
If that's too much work, you might consider option 3. I'm no expert on SOQL, so no idea what kind of work would be involved transforming ordinary SQL queries into SOQL queries. If you think it's rather algorithmic and reliable, that might be the way to go.
If you had to explain Lambda expressions to a 5th grader (10/11 years old), how would you do it? And what examples might you give, or resources might you point them to? I may be finding myself in the position of having to teach this to 5th grade level developers and could use some assistance.
[EDIT]: The "5th Grader" reference was meant to relate to an American TV show which pits adults vs. 5th graders in a quiz type setting (I think). I meant to imply that the people who need to be taught this know nothing about Lambda's and I need to find a way to make things VERY simple. I'm sorry that I forgot this forum has a worldwide audience.
Thanks very much.
Just call it a function without a name. If she has not been exposed to much programming her mind not already have been calcified in thinking that all functions should have names.
Most of the complexity related to lambda expressions is caused by complicated naming and putting it on a marble pedestal.
Lot's of kids create great websites with lot's of Javascript stuff. Chances are they are using lambda expressions all the time without knowing it. They just call it a 'cool trick'.
I don't think you need to explain lambda expressions to kids aged 10-11. Just show them how lambda expressions look like, what you can do with them, and where you can use them. Kids in that age still have the capability to learn something new without relying on analogies to understand it.
Lambda expressions are what I consider higher order programming. Rigorous explanation will require extensive prerequisite learning. Certainly, this is not practical at the 5th grade level.
However, it might help to just cover some concepts by example in a way that mirrors real life physical situations.
For instance, a scale is a sort of a lambda expression. It tallies the mass of the objects placed on it. It is not a variable because it does not store the number anywhere. Instead, it generates the number at the time of use. When used again, it recalculates based on its inputs. You can take it places and use it somewhere else, but the underlying mechanics (expression) is the same.
if he already understands what "function" is that you can say that it is the function that you need only once and therefore it doesn't need a name.
Anyway, if you need to explain functional programming I would recommend to try to steal some ideas from http://learnyouahaskell.com/ - it's one of the best explanations of ideas behind functional programming I've ever read.
Lambda expressions in c# are basically just anonymous delegates so when explaining what they are to ANYONE they need to understand in this order:
What a delegate is and what they are used for.
What an anonymous delegate is and how it is just short hand way of creating a delegate.
Lambda expression syntax and how it is just really a even more short hand way of creating an anonymous delegate.
Probably not the best thing to start explaining to a fifth grader if a cutting edge OO language (C#) didn't get it for 10 years.
Pretty hard to come up with an analogy for a function with no name... Perhaps it's significant because it's a less verbose way specify a callback?
I'm assuming you're actually looking for a basic intro for programmers, not actual 5th graders. (For 5th graders, Python or JavaScript tend to be best). Anyways, two good introductions to C# lambda expressions:
Explaining C# lambda statements in one sentence (and LINQ in about 10)
LINQ in Action
The first (disclaimer - my blog) will give you a quick explanation of the fundamental concepts. The book provides complete coverage of all relevant topics.
I am currently developing an application where you can create "programs" with it without writing source code, just click&play if you like.
Now the question is how do I generate an executable program from my data model. There are many possibilities but I am not sure which one is the best for me. I need to generate assemblies with classes and namespace and everything which can be part of the application.
CodeDOM class: I heard of lots of limitations and bugs of this class. I need to create attributes on method parameters and return values. Is this supported?
Create C# source code programmatically and then call CompileAssemblyFromFile on it: This would work since I can generate any code I want and C# supports most CLR features. But wouldn't this be slow?
Use the reflection ILGenerator class: I think with this I can generate every possible .NET code. But I think this is much more complicated and error prone than the other approaches?
Are there other possible solutions?
EDIT:
The tool is general for developing applications, it is not restricted to a specific domain. I don't know if it can be considered a visual programming language. The user can create classes, methods, method calls, all kinds of expressions. It won't be very limitating because you should be able to do most things which are allowed in real programming languages.
At the moment lots of things must still be written by the user as text, but the goal at the end is, that nearly everything can be clicked together.
You my find it is rewarding to look at the Dynamic Language Runtime which is more or less designed for creating high-level languages based on .NET.
It's perhaps also worth looking at some of the previous Stack Overflow threads on Domain Specific Languages which contain some useful links to tools for working with DSLs, which sounds a little like what you are planning although I'm still not absolutely clear from the question what exactly your aim is.
Most things "click and play" should be simple enough just to stick some pre-defined building-block objects together (probably using interfaces on the boundaries). Meaning: you might not need to do dynamic code generation - just "fake it". For example, using property-bag objects (like DataTable etc, although that isn't my first choice) for values, etc.
Another option for dynamic evaluation is the Expression class; especially in .NET 4.0, this is hugely versatile, and allows compilation to a delegate.
Do the C# source generation and don't care about speed until it matters. The C# compiler is quite quick.
When I wrote a dynamic code generator, I relied heavily on System.Reflection.Emit.
Basically, you programatically create dynamic assemblies and add new types to them. These types are constructed using the Emit constructs (properties, events, fields, etc..). When it comes to implementing methods, you'll have to use an ILGenerator to pump out MSIL op-codes into your method. That sounds super scary, but you can use a couple of tools to help:
A pre-built sample implementation
ILDasm to inspect the op-codes of the sample implementation.
It depends on your requirements, CodeDOM would certainly be the best fit for a "program" stored it in a "data model".
However its unlikely that using option 2 will be in any way measurably slower in comparision with any other approach.
I would echo others in that 1) the compiler is quick, and 2) "Click and Play" things should be simple enough so that no single widget added to a pile of widgets can make it an illegal pile.
Good luck. I'm skeptical that you can achieve point (2) for anything but really toy-level programs.
What exactly is Expression<> used for in C#? Are there any scenarios where you would instantiate Expression<>'s yourself as an object? If so, please give an example!
Thank you!
Expression<T> is almost entirely used for LINQ, but it doesn't have to be. Within LINQ, it's usually used to "capture" the logic expressed in code, but keep it in data. That data can then be examined by the LINQ provider and handled appropriately - e.g. by converting it into SQL. Usually the expression trees in LINQ are created by the compiler from lambda expressions or query expressions - but in other cases it can be handy to use the API directly yourself.
A few examples of other places I've used it and seen it used:
In MiscUtil, Marc Gravell used it to implement "generic arithmetic" - if a type has the relevant operator, it can be used generically.
In UnconstrainedMelody I used it in a similar way to perform operations on flags enums, regardless of their underlying type (which is trickier than you might expect, due to long and ulong having different ranges)
In Visual LINQ I used query expressions to "animate" LINQ, so you can see what's going on. While obviously this is a LINQ usage, it's not the traditional form of translating logic into another form.
In terms of LINQ, there are things you can do to create more versatile LINQ queries at runtime than you can purely in lambdas.
I've used Expression many times as a micro-compiler, as an alternative to DynamicMethod and IL. This approach gets stronger in .NET 4.0 (as discussed on InfoQ), but even in 3.5 there are lots of things you can do (generally based on runtime data; configuration etc):
generic operators
object cloning
complex initialization
object comparison
I also used it as part of a maths engine for some work I did with Microsoft - i.e. parse a math expression ("(x + 12) * y = z" etc) into an Expression tree, compile it and run it.
Another intersting use (illustrated by Jason Bock, here) is in genetic programming; build your candidates as Expression trees, and you have the necessary code to execute them quickly (after Compile()), but importantly (for genetic programming), also to swap fragments around.
Take a look at my before & after code in my answer to another SO question.
Summary: Expression<> greatly simplified the code, made it easier to understand, and even fixed a phantom bug.
What reason is there for C# or java having lambdas? Neither language is based around them, it appears to be another coding method to do the same thing that C# already did.
I'm not being confrontational, if there is a reason I would like to know the reason why. For the purpose of full disclosure I am a Java programmer with a C++ background with no lisp experience, I may just be missing the point.
There are common use-cases which require passing (or storing) a block of code to be executed later. The most common would be event listeners. Believe it or not, the following bit of code uses a lambda-ish construct in Java:
JButton button = new JButton("Push me!");
button.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
System.out.println("Pressed!");
}
});
The anonymous inner-class is acting as a lambda, albeit an extremely verbose one. With a little bit of implicit conversion magic, we can write the following equivalent in Scala:
val button = new JButton("Push me!")
button.addActionListener { e =>
println("Pressed!")
}
C# makes this sort of thing fairly easy with delegates and (even better) lambdas.
I see lambdas in C# as a very convenient short-cut for doing delegates. Much more readable to have the code right there where it is being used rather than having to search elsewhere for the delegate definition.
Syntactic Sugar.
It provides a convenient and more-readable way to represent an idea, in this case a tiny throw-away method. Under the hood, the compiler expands that out to a delegate and method call, but it's the thing doing the work, not you.
Lambdas allow you write less verbose, more expressive code. For example, list comprehensions...
BTW, work is under way to explore the possibility of adding closures to Java - in the meantime it is necessary to use anonymous classes instead (ugly).
C# isn't going for purity to a particular school of language design (unlike Java, which was designed by the Smalltalkers as to be something of a pure OO language). C# is going for all-things-to-all-people, and it's pretty good at it. C# is based around gathering the best of the various styles of programming into one high-quality, well-supported language. That includes procedural, object-oriented, functional, dynamic, logic, etc. styles of programming. Obviously, so far it doesn't have much in the way of dynamic or logic styles of programming, but that is soon to come (dynamic programming coming with C# 4.0).
In case of C#, lambdas are used internally to implement LINQ. See the article The Evolution Of LINQ And Its Impact On The Design Of C#
Lambda's allow for more readable code in that they allow operations to be defined closer to the point of use rather than like the current C++ method of using function objects whose definition is sometimes far from the point of use. (This is not including some of the boost libraries). I think the key point from lambdas is that they allow more concise and easy to understand code.
They offer better security using the multi threading in Java by implying in many cases the "final" option. So you are not error prone for multitasking.