Develop Reference

Run-time compilation of code in .net with better performance at compile time

before any one mark this as duplicate please try to understand what i need
I have a software created in C# ASP.NET MVC5 were user can create any formula basically if..Else,switch condition which might include some custom function also from my custom DLL which include datediff,DatePart.
So for this scenario early we were using VBScript with the help of MSScriptControl but we decided to move for C# like syntax for formula.
i have explore much like CodeDom and Roslyn but the problem is it take almost AVG 130 ms for compile of 10 line of code.
later i have also try to implement CS-Script which is much slow than above two because it internally uses CodeDOM,Roslyn,MONO as compiler option.
Now what i'm looking is alternate of VBScript which almost take 11 ms for compile and execute.
Please suggest me any other way or tool for this scenario.
How to improve compile time using CSharpCodeProvider
`using Microsoft.CSharp;
using System;
using System.CodeDom.Compiler;`
`namespace abc
{
public class Abc
{
static void Main()
{
var code = #"
using System;
namespace First
{
public class program
{
public static int Main()
{
if(Convert.ToDateTime(""01-04-2019
10:25:00"")>Convert.ToDateTime(""01-04-2019 15:00:00""))
{
return 1;
}
else
{
return 0;
}
}
}
}";
Console.WriteLine(code);
var options = new CompilerParameters();
options.GenerateExecutable = false;
options.GenerateInMemory = false;
var provider = new CSharpCodeProvider();
var compile = provider.CompileAssemblyFromSource(options, code);
var type = compile.CompiledAssembly.GetType("First.program");
var abc = Activator.CreateInstance(type);`
`var method = type.GetMethod("Main");
var result = method.Invoke(abc, null);`
`Console.WriteLine(result); //output:`
`}`
`}`
`}`

We did face some kind of similar problem in our projects. We were generating hundreds of small c# files and compile them one by one to get one function from each file (actually the file were in memory so we were not IO bound).
This operation was slow and we tried a lot of things to increase throughput (updating Roslyn, changing options, etc). We discovered that there is some overhead when starting a compilation which is not linear with the amount of code implied in it.
Our final solution which is so far the best we have (but we would really appreciate if someone has a better idea): group compilation. Calling the compiler on multiple files at once is much faster than calling it multiple times for the same amount of file.
What I would suggest in your case: group all your functions and compile them at once. 100ms for a single function is a pain when you have 1000 but 1s is ok for 1000 functions.

Best way to run a string as c# code

Let's say I have:
#{
var str= "DateTime.Now";
}
I want to process this string as a c# code
#Html.Raw(App.ProcessAsCode(str));
The output should be the current date time.

Final Edit:
Based on further information - if the goal here is to simply have a formatting engine there are lots of options out there. One such option is based around the .liquid syntax from shopify (see here). You can find a .NET port of this on gitHub here: https://github.com/formosatek/dotliquid/. The main purpose of this is to turn something like:
<h2>{{product.name}}</h2>
Into something like:
<h2>Beef Jerky</h2>
I would strongly recommend reading more about the liquid engine and syntax and I believe this will lead you in the right direction. Best of luck!
Initial Answer
This is definitely possible - although as others have said you will want to be careful in what you do. Using C# the key to compiling and running code generically is the "CSharpCodeProvider" class. Here is a brief example of how that looks:
string[] references = { "System.dll" };
CompilerParams.ReferencedAssemblies.AddRange(references);
var provider = new CSharpCodeProvider();
CompilerResults compile = provider.CompileAssemblyFromSource(CompilerParams, formattedCode);
In this example, "formattedCode" is a string with the C# code. Any references must be manually added. For the full example see this stack question (How to get a Type from a C# type name string?).
NOTE -- If all you are looking to do here is a format string or something simple like that you might have the user pass in a .NET format string (eg "MM/dd/yyyy"), then use that in a call to the "ToString" method. That would provide the user some configurability, while still making sure your system stays secure. In general running code on a server that hasn't been properly checked/escaped is really dangerous!
Reference - For your reference, the current msdn page for CSharpCodeProvider also has some examples.

Another option would be using a dynamic language such as IronRuby or IronPython.

How to access project code meta data?

In my VSPackage I need to replace reference to a property in code with its actual value. For example
public static void Main(string[] args) {
Console.WriteLine(Resource.HelloWorld);
}
What I want is to replace "Resource.HelloWorld" with its actual value - that is, find class Resource and get value of its static property HelloWorld. Does Visual Studio expose any API to handle code model of the project? It definitely has one, because this is very similar to common task of renaming variables. I don't want to use reflection on output assembly, because it's slow and it locks the file for a while.

There is no straight forward way to do this that I know of. Reliably getting an AST out of Visual Studio (and changes to it) has always been a big problem. Part of the goal of the Rosalyn project is to create an unified way of doing this, because many tool windows had their own way of doing this sort of stuff.
There are four ways to do this:
Symbols
FileCodeModel + CodeDOM
Rosalyn AST
Unexplored Method
Symbols
I believe most tool windows such as the CodeView and things like Code Element Search use the symbols created from a compiled build. This is not ideal as it is a little more heavy weight and hard to keep in sync. You'd have to cache symbols to make this not slow. Using reflector, you can see how CodeView implements this.
This approach uses private assemblies. The code for getting the symbols would look something like this:
var compilerHost = new IDECompilerHost();
var typeEnumerator = (from compiler in compilerHost.Compilers.Cast<IDECompiler>()
from type in compiler.GetCompilation().MainAssembly.Types
select new Tuple<IDECompiler, CSharpType>(compiler, type));
foreach (var typeTuple in typeEnumerator)
{
Trace.WriteLine(typeTuple.Item2.Name);
var csType = typeTuple.Item2;
foreach (var loc in csType.SourceLocations)
{
var file = loc.FileName.Value;
var line = loc.Position.Line;
var charPos = loc.Position.Character;
}
}
FileCodeModel + CodeDOM
You could try using the EnvDTE service to get the FileCodeModel associated with a Code Document. This will let you get classes and methods. But it does not support getting the method body. You're messing with buggy COM. This ugly because an COM object reference to a CodeFunction or CodeClass can get invalided without you knowing it, meaning you'd have to keep your own mirror.
Rosalyn AST
This allows provides the same capabilities as both FileCodeModel and Symbols. I've been playing with this and it's actually not too bad.
Unexplored Method
You could try getting the underlying LanguageServiceProvider that is associated with the Code Document. But this is really difficult to pull off, and leaves you with many issues.

Why is JIT_MethodAccessAllowedBySecurity taking so much time?

I'm working on a C# application that allows users to basically import tables of data, and then enter their own formulas in a mini-language to compute new columns from the underlying data.
These formulas are compiled into LINQ expression trees in the engine, which the .NET 4.0 expression tree library then presumably compiles into IL so they can be executed.
We've started using our engine for some high-volume ticking data recently, and we're finding the speed of these compiled expression trees to be a real bottleneck - the speed is pretty slow when re-calculating all these columns on the fly. Hitting it with the built-in Visual Studio 2010 profiler reveals half of all our execution time is being spent in clr.dll, in a method called JIT_MethodAccessAllowedBySecurity.
Cursory googling of this string hasn't yielded anything, so I'm wondering if there's anyone out there who can tell me what this method is, and whether there's a way to keep it from eating up all my cycles? Maybe there's a way to compile this code and explicitly give it permission to do whatever it wants so the clr can stop these checks? Perhaps the temporary assemblies being generated by the expression tree engine do not have full trust?
Anyhow, I'm pretty much at a loss and I'm very interested to hear if any other StackOverflow'ers have come across this issue in the past. Thanks in advance!

The solution is to use LambdaExpression.CompileToMethod(MethodBuilder method) instead of LambdaExpression.Compile().
I think Jethro was on the right track when he posited that CAS was involved. In testing, the profiler only started showing calls to JIT_MethodAccessAllowedBySecurity when I used expression trees to call functions that weren't dynamically defined in the generated assembly (i.e. using Expression.Call to invoke a library method rather than a piece of generated code.) This suggests that the slowdown was caused by CAS checking that my generated code had access to the methods it was invoking. It seems to follow that by applying declarative security modifications to the functions I wished to call, I could avoid this overhead.
Unfortunately, I wasn't able to get rid of the JIT_MethodAccessAllowedBySecurity overhead through any use of declarative security (PermissionSet, SecurityAction.LinkDemand, and the like). At one point I had literally every method in my project marked with [PermissionSet(SecurityAction.LinkDemand, Unrestricted = true)], with no results.
Luckily, when looking for ways to add attributes to the generated delegates, I stumbled upon the solution - using a MethodBuilder to compile the expression tree rather than the built-in LambdaExpression.Compile method.
I've included the bit of code that replaced .Compile() and led to elimination of the JIT_MethodAccessAllowedBySecurity calls and a >2x speedup in our calculation engine:
// T must be of delegate type (Func<T>, Func<T1, T2>, etc.)
public static T GetCompiledDelegate<T>(Expression<T> expr)
{
var assemblyName = new AssemblyName("DelegateHostAssembly") { Version = new Version("1.0.0.0") };
var assemblyBuilder =
AppDomain.CurrentDomain.DefineDynamicAssembly(
assemblyName,
AssemblyBuilderAccess.RunAndSave);
var moduleBuilder = assemblyBuilder.DefineDynamicModule("DelegateHostAssembly", "DelegateHostAssembly.dll");
var typeBuilder = moduleBuilder.DefineType("DelegateHostAssembly." + "foo", TypeAttributes.Public);
var methBldr = typeBuilder.DefineMethod("Execute", MethodAttributes.Public | MethodAttributes.Static);
expr.CompileToMethod(methBldr);
Type myType = typeBuilder.CreateType();
var mi = myType.GetMethod("Execute");
// have to box to object because .NET doesn't allow Delegates as generic constraints,
// nor does it allow casting of Delegates to generic type variables like "T"
object foo = Delegate.CreateDelegate(typeof(T), mi);
return (T)foo;
}
This code is consistently >2x faster when using any code that uses Expression trees to call functions that are not themselves defined by Expression trees. Thanks for everyone's help, and I hope this saves someone else some cycles.

I think this has something to do with CAS (Code access security).
CAS is assembly based. When you code
calls a protected method, the .NET
framework runtimes checks your
assembly to see if it has been granted
on or more permissions necessary for
the method. The .NET Framework rutime
then walks the stack to check every
assembly in the stack for these
ermissions. If one assembly does not
have all of the required permissions,
a securityexception is raised and the
code is run.
The below is what I think is happening to your code.
... a stack walk occurs and policy
check is preformed every time that the
method is called. This is a particular
problem for components in a class
library, which may be called many
times. In this situation, you can use
link demand to indicate that the
permission set check is performed on
at link time as part of the JIT
compliction process. To do this, you
decorate the method by using a
permission attribute that has a
parameter of the value
SecurityAction.LinkDemand.
I hope this helps, it looks like all you need to do is set the SecurityAction.LinkDemand attribute. The quoted text comes from Advanced foundations of Microsoft .NET 2.0 Development.
Regards

How will you use the C# 4 dynamic type?

C# 4 will contain a new dynamic keyword that will bring dynamic language features into C#.
How do you plan to use it in your own code, what pattern would you propose ? In which part of your current project will it make your code cleaner or simpler, or enable things you could simply not do (outside of the obvious interop with dynamic languages like IronRuby or IronPython)?
PS : Please if you don't like this C# 4 addition, avoid to bloat comments negatively.
Edit : refocussing the question.
The classic usages of dynamic are well known by most of stackoverflow C# users. What I want to know is if you think of specific new C# patterns where dynamic can be usefully leveraged without losing too much of C# spirit.

Wherever old-fashioned reflection is used now and code readability has been impaired. And, as you say, some Interop scenarios (I occasionally work with COM).
That's pretty much it. If dynamic usage can be avoided, it should be avoided. Compile time checking, performance, etc.
A few weeks ago, I remembered this article. When I first read it, I was frankly appalled. But what I hadn't realised is that I didn't know how to even use an operator on some unknown type. I started wondering what the generated code would be for something like this:
dynamic c = 10;
int b = c * c;
Using regular reflection, you can't use defined operators. It generated quite a bit of code, using some stuff from a Microsoft namespace. Let's just say the above code is a lot easier to read :) It's nice that it works, but it was also very slow: about 10,000 times slower than a regular multiplication (doh), and about 100 times slower than an ICalculator interface with a Multiply method.
Edit - generated code, for those interested:
if (<Test>o__SiteContainer0.<>p__Sitea == null)
<Test>o__SiteContainer0.<>p__Sitea =
CallSite<Func<CallSite, object, object, object>>.Create(
new CSharpBinaryOperationBinder(ExpressionType.Multiply,
false, false, new CSharpArgumentInfo[] {
new CSharpArgumentInfo(CSharpArgumentInfoFlags.None, null),
new CSharpArgumentInfo(CSharpArgumentInfoFlags.None, null) }));
b = <Test>o__SiteContainer0.<>p__Site9.Target(
<Test>o__SiteContainer0.<>p__Site9,
<Test>o__SiteContainer0.<>p__Sitea.Target(
<Test>o__SiteContainer0.<>p__Sitea, c, c));

The dynamic keyword is all about simplifying the code required for two scenarios:
C# to COM interop
C# to dynamic language (JavaScript, etc.) interop
While it could be used outside of those scenarios, it probably shouldn't be.

Recently I have blogged about dynamic types in C# 4.0 and among others I mentioned some of its potential uses as well as some of its pitfalls. The article itself is a bit too big to fit in here, but you can read it in full at this address.
As a summary, here are a few useful use cases (except the obvious one of interoping with COM libraries and dynamic languages like IronPython):
reading a random XML or JSON into a dynamic C# object. The .Net framework contains classes and attributes for easily deserializing XML and JSON documents into C# objects, but only if their structure is static. If they are dynamic and you need to discover their fields at runtime, they can could only be deserialized into dynamic objects. .Net does not offer this functionality by default, but it can be done by 3rd party tools like jsonfx or DynamicJson
return anonymous types from methods. Anonymous types have their scope constrained to the method where they are defined, but that can be overcome with the help of dynamic. Of course, this is a dangerous thing to do, since you will be exposing objects with a dynamic structure (with no compile time checking), but it might be useful in some cases. For example the following method reads only two columns from a DB table using Linq to SQL and returns the result:
public static List<dynamic> GetEmployees()
{
List<Employee> source = GenerateEmployeeCollection();
var queyResult = from employee in source
where employee.Age > 20
select new { employee.FirstName, employee.Age };
return queyResult.ToList<dynamic>();
}
create REST WCF services that returns dynamic data. That might be useful in the following scenario. Consider that you have a web method that returns user related data. However, your service exposes quite a lot of info about users and it will not be efficient to just return all of them all of the time. It would be better if you would be able to allow consumers to specify the fields that they actually need, like with the following URL
http://api.example.com/users?userId=xxxx&fields=firstName,lastName,age
The problem then comes from the fact that WCF will only return to clients responses made out of serialized objects. If the objects are static then there would be no way to return dynamic responses so dynamic types need to be used. There is however one last problem in here and that is that by default dynamic types are not serializable. In the article there is a code sample that shows how to overcome this (again, I am not posting it here because of its size).
In the end, you might notice that two of the use cases I mentioned require some workarounds or 3rd party tools. This makes me think that while the .Net team has added a very cool feature to the framework, they might have only added it with COM and dynamic languages interop in mind. That would be a shame because dynamic languages have some strong advantages and providing them on a platform that combines them with the strengths of strong typed languages would probably put .Net and C# ahead of the other development platforms.

Miguel de Icaza presented a very cool use case on his blog, here (source included):
dynamic d = new PInvoke ("libc");
d.printf ("I have been clicked %d times", times);
If it is possible to do this in a safe and reliable way, that would be awesome for native code interop.

This will also allow us to avoid having to use the visitor pattern in certain cases as multi-dispatch will now be possible
public class MySpecialFunctions
{
public void Execute(int x) {...}
public void Execute(string x) {...}
public void Execute(long x) {...}
}
dynamic x = getx();
var myFunc = new MySpecialFunctions();
myFunc.Execute(x);
...will call the best method match at runtime, instead of being worked out at compile time

I will use it to simplify my code which deals with COM/Interop where before I had to specify the member to invoke, its parameters etc. (basically where the compiler didn't know about the existence of a function and I needed to describe it at compile time). With dynamic this gets less cumbersome and the code gets leaner.