I'm looking to see if there is a method or tool to view how things like closures or query expressions are created by the c# compiler "under the hood". I've noticed that many blog posts dealing with these issues will have the original code with syntactic sugar and the underlying c# code that the compiler converts that to. So for example with linq and query expressions they would show:
var query = from x in myList select x.ToString();
then the resulting code would be
var query = myList.Select(x=>x.ToString());
Is it possible with a tool or do you just have to know how it works from the spec and go from there?
Resharper can do this conversion (LINQ expression syntax to lambda syntax) for you very easily.
LINQPad has a tab that can show you the lambda expression syntax for a query you enter into it, and it has another tab that disassembles it all the way down to the IL code level. (There's another tab that shows you the SQL that gets generated if you're using LINQ to SQL or LINQ to Entities).
SharpLab.io is the tool you are looking for.
From the site, SharpLab shows intermediate steps and results of code compilation. It allows you to see the code as compiler sees it, and get a better understanding of .NET languages.
I'm not a user of Resharper, but I'm sure that will work.
You already seem to know the alternate form of your Linq syntax query. I think that is the answer; Know what you are doing. Read up on it. If you don't know, you are dangerous, and you will go nowhere near my code base. :)
Seriously, this takes a minimum of reading up on, and provides you with immense power in C#. If you are at all interested in your work, you know this.
How far under the hood do you want to go?
If you're really interested in seeing what your code looks like after you compile it, you should check out ildasm.exe. This tool will show you the cold, hard, IL, generated when you compile your application. This tool will allow you to open up any of your compiled assemblies are view the real nuts and bolts that are under the hood.
Related
I am trying to implement a DSL like feature in C#. It may look something similar to LINQ queries. I am wondering if it is possible to implement new unary or binary operators using Roslyn.
I have been googling last few days without much success. It would be great if someone could point me to some samples or Roslyn documentations.
There are two ways how you could use Roslyn to implement a new C#-based language.
Use the Roslyn API to parse the source code into a syntax tree, then transform the syntax tree into actual C# and compile that.
This is ideal if your language is actually syntactically valid C# code, but the semantics are different. For example, you could implement await this way, if you forced await to look like a function call (e.g. await(x) would be valid, but not await x).
If you want to introduce new syntax (like a new operator), it might work, since Roslyn does support parsing “broken” code. But it most likely won't work that well, because then the syntax tree might not look the way you want. Worse, the results might not be consistent (sometimes, your new syntax will be parsed one way, sometimes another).
Since Roslyn is now open source, you can actually modify the source code of the compiler in any way you want, including adding a new operator.
But doing that is most likely not going to be simple. And I think the workflow is also going to be more complicated: you need to compile your own version of the compiler, not just use a library from NuGet like in option 1.
Does anyone know if there is a way to evaluate c# code at runtime.
eg. I would like to allow a user to enter DateTime.Now.AddDays(1), or something similar, as a string and then evaluate the string to get the result.
I woder if it is possible to access the emmediate windows functionality, since it seems that is evaluates every line entered dynamically.
I have found that VB has an undocumented EbExecuteLine() API function from the VBA*.dll and wonder if there is something equivalent for c#.
I have also found a custom tool https://github.com/DavidWynne/CSharpEval (it used to be at kamimucode.com but the author has moved it to GitHub) that seems to do it, but I would prefer something that comes as part of .NET
Thanks
Mono has the interactive command line (csharp.exe)
You can look at it's source code to see exactly how it does it's magic:
https://github.com/mono/mono/raw/master/mcs/tools/csharp/repl.cs
As you've probably already seen, there is no built-in method for evaluating C# code at runtime. This is the primary reason that the custom tool you mentioned exists.
I also have a C# eval program that allows for evaluating C# code. It provides for evaluating C# code at runtime and supports many C# statements. In fact, this code is usable within any .NET project, however, it is limited to using C# syntax. Have a look at my website, http://csharp-eval.com, for additional details.
Microsoft's C# compiler don't have Compiler-as-a-Service yet (Should come with C# 5.0).
You can either use Mono's REPL, or write your own service using CodeDOM
Its not fast but you can compile the code on the fly, see my previous question,
Once you have the assembly and you know the type name you can construct an instance of your compiled class using reflection and execute your method..
The O2 Platform's C# REPL Script Environment use the Fluent# APIs which have a real powerful reflection API that allows you do execute code snippets.
For example:
"return DateTime.Now.ToLongTimeString();".executeCodeSnippet();
will return
5:01:22 AM
note that the "...".executeCodeSnippet(); can actually execute any valid C# code snippet (so it is quite powerful).
If you want to control what your users can execute, I could use AST trees to limite the C# features that they have access to.
Also take a look at the Microsoft's Roslyn, which is VERY powerful as you can see on Multiple Roslyn based tools (all running Stand-Alone outside VisualStudio)
I'm making a Genetic Program, but I'm hitting a limitation with C# where I want to present new functions to the algorithm but I can't do it without recompiling the program. In essence I want the user of the program to provide the allowed functions and the GP will automatically use them. It would be great if the user is required to know as little about programming as possible.
I want to plug in the new functions without compiling them into the program. In Python this is easy, since it's all interpreted, but I have no clue how to do it with C#. Does anybody know how to achieve this in C#? Are there any libraries, techniques, etc?
It depends on how you want the user of the program to "provide the allowed functions."
If the user is choosing functions that you've already implemented, you can pass these around as delegates or expression trees.
If the user is going to write their own methods in C# or another .NET language, and compile them into an assembly, you can load them using Reflection.
If you want the user to be able to type C# source code into your program, you can compile that using CodeDom, then call the resulting assembly using Reflection.
If you want to provide a custom expression language for the user, e.g. a simple mathematical language, then (assuming you can parse the language) you can use Reflection.Emit to generate a dynamic assembly and call that using -- you guessed it -- Reflection. Or you can construct an expression tree from the user code and compile that using LINQ -- depends on how much flexibility you need. (And if you can afford to wait, expression trees in .NET 4.0 remove many of the limitations that were in 3.5, so you may be able to avoid Reflection.Emit altogether.)
If you are happy for the user to enter expressions using Python, Ruby or another DLR language, you can host the Dynamic Language Runtime, which will interpret the user's code for you.
Hosting the DLR (and IronPython or IronRuby) could be a good choice here because you get a well tested environment and all the optimisations the DLR provides. Here's a how-to using IronPython.
Added in response to your performance question: The DLR is reasonably smart about optimisation. It doesn't blindly re-interpret the source code every time: once it has transformed the source code (or, specifically, a given function or class) to MSIL, it will keep reusing that compiled representation until the source code changes (e.g. the function is redefined). So if the user keeps using the same function but over different data sets, then as long as you can keep the same ScriptScope around, you should get decent perf; ditto if your concern is just that you're going to run the same function zillions of times during the genetic algorithm. Hosting the DLR is pretty easy to do, so it shouldn't be hard to do a proof of concept and measure to see if it's up to your needs.
You can try to create and manipulate Expression Trees. Use Linq to evaluate expression trees.
You can also use CodeDom to compile and run a function.
For sure you can google to see some examples that might fit your needs.
It seems that this article "How to dynamically compile C# code" and this article "Dynamically executing code in .Net" could help you.
You have access to the Compiler from within code, you can then create instances of the compiled code and use them without restarting the application. There are examples of it around
Here
and
Here
The second one is a javascript evaluator but could be adapted easily enough.
You can take a look at System.Reflection.Emit to generate code at the IL level.
Or generate C#, compile into a library and load that dynamically. Not nearly as flexible.
It is in fact very easy to generate IL. See this tutorial: http://www.meta-alternative.net/calc.pdf
What exactly is Expression<> used for in C#? Are there any scenarios where you would instantiate Expression<>'s yourself as an object? If so, please give an example!
Thank you!
Expression<T> is almost entirely used for LINQ, but it doesn't have to be. Within LINQ, it's usually used to "capture" the logic expressed in code, but keep it in data. That data can then be examined by the LINQ provider and handled appropriately - e.g. by converting it into SQL. Usually the expression trees in LINQ are created by the compiler from lambda expressions or query expressions - but in other cases it can be handy to use the API directly yourself.
A few examples of other places I've used it and seen it used:
In MiscUtil, Marc Gravell used it to implement "generic arithmetic" - if a type has the relevant operator, it can be used generically.
In UnconstrainedMelody I used it in a similar way to perform operations on flags enums, regardless of their underlying type (which is trickier than you might expect, due to long and ulong having different ranges)
In Visual LINQ I used query expressions to "animate" LINQ, so you can see what's going on. While obviously this is a LINQ usage, it's not the traditional form of translating logic into another form.
In terms of LINQ, there are things you can do to create more versatile LINQ queries at runtime than you can purely in lambdas.
I've used Expression many times as a micro-compiler, as an alternative to DynamicMethod and IL. This approach gets stronger in .NET 4.0 (as discussed on InfoQ), but even in 3.5 there are lots of things you can do (generally based on runtime data; configuration etc):
generic operators
object cloning
complex initialization
object comparison
I also used it as part of a maths engine for some work I did with Microsoft - i.e. parse a math expression ("(x + 12) * y = z" etc) into an Expression tree, compile it and run it.
Another intersting use (illustrated by Jason Bock, here) is in genetic programming; build your candidates as Expression trees, and you have the necessary code to execute them quickly (after Compile()), but importantly (for genetic programming), also to swap fragments around.
Take a look at my before & after code in my answer to another SO question.
Summary: Expression<> greatly simplified the code, made it easier to understand, and even fixed a phantom bug.
As a fairly junior developer, I'm running into a problem that highlights my lack of experience and the holes in my knowledge. Please excuse me if the preamble here is too long.
I find myself on a project that involves my needing to learn a number of new (to me) technologies, including LINQ (to OBJECTS and to XML for purposes of this project) among others. Everything I've read to this point suggests that to utilize LINQ I'll need to fully understand the following (Delegates, Anonymous Methods and Lambda Expressions).
OK, so now comes the fun. I've CONSUMED delegates in the past as I have worked with the .NET event model, but the majority of the details have been hidden from me (thanks Microsoft!). I understand that on a basic level, delegate instances are pointers to methods (a gross over-simplification, I know).
I understand that an anonymous method is essentially an in-line unnamed method generally (if not exclusively) created as a target for a delegate.
I also understand that lambdas are used in varying ways to simplfy syntax and can be used to point a simple anonymous method to a delegate.
Pardon me if my any of my descriptions are WAY off here, this is the basic level to which I understand these topics.
So, the challenge:
Can anyone tell me if at least on a basic level if my understanding of these items is even close? I'm not looking for complex esoteric minutiae, just the basics (for now).
To what degree do I need to truly understand these concepts before applying LINQ in a project to reasonable effect? I want to understand it fully and am willing to spend the time. I simply may not HAVE the time to fully grok all of this stuff before I need to produce some work.
Can anyone point me to some good articles that explain these subjects and apply them to "real world" examples so that I can get my head around the basics of the topics and application of them? What I mean by real world, is how might I use this in the context of "Customers and Invoices" rather than abstract "Vectors and Shapes" or "Animals and Cows". The scenario can be somewhat contrived for demonstration purposes, but hopefully not strictly academic. I have found a number of examples on-line and in books, but few seem to be "Plain English" explanations.
Thank you all in advance for your patience, time and expertise.
Where can i find a good in depth guide to C# 3?
1) Your knowledge so far seems ok. Lambda expressions are turned into anonymous methods or System.Linq.Expressions.Expression's, depending on context. Since you aren't using a database technology, you don't need to understand expressions (all lambdas will be anonymous methods). You didn't list Extension methods, but those are very important (and easy) to understand. Make sure you see how to apply an extension method to an interface - as all the functionality in linq comes from System.Linq.Enumerable - a collection of extention methods against IEnumerable(Of T).
2) You don't need a deep understanding of lambdas.
The arrow syntax ( => ) was the biggest hurdle for me. The arrow separates the signature and the body of the lambda expression.
Always remember : Linq methods are not executed until enumerated.
Watch out for using loop variables in a lambda. This is a side effect from deferred execution that is particularly tricky to track down.
3) Sure, Here are some of my answers that show linq method calls - some with xml.
List splitting
Simple Xml existence search
Xml projection - shape change
1) Those descriptions sound pretty accurate to me. Sometimes anonymous methods and lambda expressions will need to create a new type to put the target of the delegate in, so they can act as closures.
2/3) I would read up a bit until you're happy with delegates, anonymous methods and lambda expressions. I dedicate a chapter to the delegate-related changes in each of C# 2.0 and C# 3.0 in C# in Depth, although of course other books go into detail too. I have an article as well, if that helps.
As for examples - delegates are used for many different purposes. They're all different ways of looking at the same functionality, but they can feel very different:
Providing the code to call when you start a new thread
Reacting to UI events
Providing the filter, selection, ordering etc for a LINQ query
Providing a callback for when an asynchronous operation has finished
If you have any specific situations you'd like an example of, that would be easier to answer.
EDIT: I should point out that it's good news that you're only working with LINQ to Objects and LINQ to XML at the moment, as that means you don't need to understand expression trees yet. (They're cool, but one step at a time...) LINQ to XML is really just an XML API which works nicely with LINQ - from what I remember, the only times you'll use delegates with LINQ to XML are when you're actually calling into LINQ to Objects. (That's very nice to do, admittedly - but it means you can reuse what you've already learned.)
As you've already got C# in Depth, chapters 10 and 11 provide quite a few examples of using lambda expressions (and query expressions which are translated into lambda expressions) in LINQ. Chapter 5 has a few different examples of delegate use.
Read this...
http://linqinaction.net/
..and all you're question will be answered!!!