I was having trouble with performance when using Roslyn to compile to a dynamic assembly. Compilation was taking ~3 seconds, compared to ~300 milliseconds to compile the same code when using the CodeDom compiler. Here's a pared-down version of the code I'm using to do the compilation:
var compilation = CSharpCompilation.Create(
"UserPayRules.dll",
syntaxTrees,
assembliesToAdd);
using (var stream = new MemoryStream())
{
stopWatch.Start();
var result = compilation.Emit(stream);
stopWatch.Stop();
Debug.WriteLine("Compilation: {0}", stopWatch.ElapsedMilliseconds);
if (!result.Success)
{
throw new InvalidOperationException();
}
var assembly = Assembly.Load(stream.GetBuffer());
}
This answer suggests passing a ModuleBuilder object into the Emit method instead of a MemoryStream in order to speed things up. I tried to follow that pattern, like so:
var compilation = CSharpCompilation.Create(
"UserPayRules.dll",
syntaxTrees,
assembliesToAdd);
var assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(
new AssemblyName("ThisAssembly"),
AssemblyBuilderAccess.RunAndCollect);
var moduleBuilder = assemblyBuilder.DefineDynamicModule("ThisModule");
var result = compilation.Emit(moduleBuilder);
Debug.WriteLine("Compilation: {0}", stopWatch.ElapsedMilliseconds);
if (!result.Success)
{
throw new InvalidOperationException();
}
var assembly = Assembly.Load(stream.GetBuffer());
But my version of Roslyn apparently doesn't have an overload of the Emit method that takes a ModuleBuilder. That version is:
Id: Microsoft.CodeAnalysis
Version: 0.6.4033103-beta (Prerelease)
Project Information: http://msdn.microsoft.com/en-US/roslyn
Obviously, this is a prerelease, so it's not strange that the api might have changed. However,
My Question(s)
Does anyone know why the Emit method no longer seems to have an overload that takes a ModuleBuilder?
Is there another way to make this compilation faster while still using Roslyn (Roslyn offers a couple advantages over the CodeDom and Mono compilers that I'd prefer not to give up)?
Roslyn currently doesn't expose ability to emit dynamic assemblies. We removed it because it was problematic.
You can still emit to a MemoryStream using Compilation.Emit APIs and then use Assembly.Load(byte[]) to load the resulting binary.
Note that this assembly won't be freed until the containing AppDomain is unloaded.
Related
This question is based in (ILSpy, how to resolve dependencies?)
Well, the ILSpy library changed and now we don't have Ast Builder, like the most answers about this.
The instance new ICSharpDecompiler have a overload that can take a ModuleDefinition of Mono.Cecil, but I don't know if this ModuleDefinition is the new "Resolver" for this issue.
In the resume I need to decompile a assembly (C#, .dll) that I don't have the dependecies (dll), I need to read the assembly using CSharpDecompiler decompiler = new CSharpDecompiler(dll.Name + ".dll", new DecompilerSettings()); without have the dependencies, how can I do that?
I got the solution making a simple:
CSharpDecompiler decompiler = new CSharpDecompiler("name.dll", new DecompilerSettings() {ThrowOnAssemblyResolveErrors = false})
This will ignore the dependencies but the types with references will not be read.
I am enhancing an existing process with runtime code generation. The code I am creating at runtime needs access to some of the same dlls that the process creating the code is already referencing.
Problem is that the process runs within some 3rd party software that loads dlls from resources and injects them into my process... thus I do not have access to either a dll on disk nor to the resource that contained the dll in the external wrapper.
As a result I am trying to use the assemblies I already have in memory and feed them into the Roslyn workspace into which I place my runtime code for compilation.
I thought I could try serializing the Assembly with a binary formatter as per this SO:
Opposite operation to Assembly Load(byte[] rawAssembly)
But even if I pretty much take the code as is:
Assembly yourAssembly = typeof(object).Assembly;
var formatter = new BinaryFormatter();
var ms = new MemoryStream();
formatter.Serialize(ms, yourAssembly);
var reloadedAssembly = Assembly.Load(ms.GetBuffer());
I get:
An exception of type 'System.BadImageFormatException' occurred in mscorlib.dll but was not handled in user code
None of the other search results seemed any better.
What I want to do is something like:
var assemblyRef = MetadataReference.CreateFromAssembly(typeof(object).Assembly);
mySolution.AddMetadataReference(projectId, assemblyRef);
Any suggestions?
For a managed assembly loaded using Assembly.Load(byte[]) you can create a Roslyn MetadataReference like so:
var assembly = Assembly.Load(bytes);
var modulePtr = Marshal.GetHINSTANCE(assembly.ManifestModule);
var peReader = new PEReader((byte*)modulePtr, bytes.Length))
var metadataBlock = peReader.GetMetadata();
var moduleMetadata = ModuleMetadata.CreateFromMetadata((IntPtr)metadataBlock.Pointer, metadataBlock.Length);
var assemblyMetadata = AssemblyMetadata.Create(moduleMetadata);
var reference = assemblyMetadata.GetReference();
Note that this doesn't work for assemblies loaded from a file, since the layout in the memory is different.
ILSpy is using cecil along with it's own libraries to accomplish this, i have heard that .NETReflector is using his own private algorithm to generate the higher level c#,f#,vb code from MSIL, any methods/references on how to do this without dependencies ?
EDIT: i'm now satified with ILSpy and cecil as a backbone
SOLUTION:
string GetSourceCode(string path) {
var asmDef = AssemblyDefinition.ReadAssembly(path);
var strBuilder = new StringBuilder();
foreach (var type in asmDef.MainModule.Types) {
if (!type.IsCompilerGenerated()) {
AstBuilder builder = new AstBuilder(new DecompilerContext(asmDef.MainModule));
builder.AddType(type);
var output = new StringWriter();
builder.GenerateCode(new PlainTextOutput(output));
strBuilder.Append(output.ToString());
output.Dispose();
}
}
return strBuilder.ToString();
}
Well, first thing you have to understand, is that there is no such thing as plain source code in compiled dll or exe file. Upon compilation C# code is compiled into CIL. Mono.Cecil reads dll or exe file as byte array and decodes it into CIL instructions. The same thing does .Net platform, when it runs your programm. What ILSpy does is that it tries to guess how did the source code look like, based upon decoded instructions.
.NETReflector does the same thing but without using Cecil
While working with latest version of roslyn-ctp I have found that it does not support dynamic keyword while compiling and script execution, i.e. you will get an compiling error error CS8000: This language feature ('dynamic') is not yet implemented in Roslyn. Here is a short code snippet:
var engine = new ScriptEngine();
var script = #"dynamic someVariable = 0;";
// you an error CS8000: This language feature ('dynamic') is not yet implemented in Roslyn
engine.CreateSession().Execute(script);
While working with await keyword…
In contrast, while working with await keyword at compilation or script, I usually got some random compilation error like one of followings:
error CS1001: Identifier expected
error CS1003: Syntax error, ',' expected
error CS0246: The type or namespace name 'await' could not be found (are you missing a using directive or an assembly reference?)
Sample of scripting
var engine = new ScriptEngine();
new[]
{
"System", "System.Threading", "System.Threading.Tasks",
} .ToList().ForEach(#namespace => engine.ImportNamespace(#namespace));
var script = #"await Task.Run(() => System.Console.WriteLine(""Universal [async] answer is '42'""));";
engine.CreateSession().Execute(script);
Sample of compilation
// compilation sample
const string codeSnippet = #"namespace DemoNamespace
{
using System;
using System.Threading;
using System.Threading.Tasks;
public class Printer
{
public async void Answer()
{
var answer = 42;
var task = Task.Run(() => System.Console.WriteLine(string.Format(""Universal [async] answer is '{0}'"", answer)));
await task; // not working
task.Wait(); // working as expected
}
}
}";
var syntaxTree = SyntaxTree.ParseText(codeSnippet,
options: new ParseOptions(languageVersion: LanguageVersion.CSharp5));
var references = new []
{
MetadataReference.CreateAssemblyReference(typeof(Console).Assembly.FullName),
MetadataReference.CreateAssemblyReference(typeof(System.Threading.Tasks.Task).Assembly.FullName),
};
var compilation = Compilation.Create(
outputName: "Demo",
options: new CompilationOptions(OutputKind.DynamicallyLinkedLibrary),
syntaxTrees: new[] { syntaxTree },
references: references);
if(compilation.GetDiagnostics().Any())
{
compilation.GetDiagnostics().Select(diagnostic => diagnostic).Dump();
throw new Exception("Compilation failed");
}
Assembly compiledAssembly;
using (var stream = new MemoryStream())
{
EmitResult compileResult = compilation.Emit(stream);
compiledAssembly = Assembly.Load(stream.GetBuffer());
}
dynamic instance = Activator.CreateInstance(compiledAssembly.GetTypes().First());
instance.Answer();
Q: Am I missing something or it is not implemented yet?
I have tried different configuration with LanguageVersion.CSharp5 and without. Both Google and Stackoverflow searches are full of marketing hype for both roslyn and async keywords and almost useless. Microsoft "Roslyn" CTP forum also has no answer for this.
ps: as far as I know async keyword has introduced for readability both by humans and compilers while await does all magic
await support is not implemented in the current Roslyn CTP (although it is now implemented in internal builds).
The reason for the difference in error reporting is that we first built the Roslyn parser so that it could handle the entire C# 4 language, and then filled in semantics for features one at a time. Since await is a C# 5 feature, it is not even recognized by the parser, and there is no place to recognize its use and provide a good error.
Actually, the Roslyn forum does have the answer. If you look at the post Known Limitations and Unimplemented Language Features, you'll notice that it contains “Async” among the not yet implemented features in C#.
That list is about the June CTP, but since the list of changes between the June CTP and the December CTP doesn't list async, it means it's simply not implemented yet.
I think the reason for the difference in error message is that Roslyn does understand dynamic (but doesn't implement it yet). On the other hand, it doesn't understand async-await, so it gives you generic compilation errors.
I have some C# code which is using CSharpCodeProvider.CompileAssemblyFromSource to create an assembly in memory. After the assembly has been garbage collected, my application uses more memory than it did before creating the assembly. My code is in a ASP.NET web app, but I've duplicated this problem in a WinForm. I'm using System.GC.GetTotalMemory(true) and Red Gate ANTS Memory Profiler to measure the growth (about 600 bytes with the sample code).
From the searching I've done, it sounds like the leak comes from the creation of new types, not really from any objects that I'm holding references to. Some of the web pages I've found have mentioned something about AppDomain, but I don't understand. Can someone explain what's going on here and how to fix it?
Here's some sample code for leaking:
private void leak()
{
CSharpCodeProvider codeProvider = new CSharpCodeProvider();
CompilerParameters parameters = new CompilerParameters();
parameters.GenerateInMemory = true;
parameters.GenerateExecutable = false;
parameters.ReferencedAssemblies.Add("system.dll");
string sourceCode = "using System;\r\n";
sourceCode += "public class HelloWord {\r\n";
sourceCode += " public HelloWord() {\r\n";
sourceCode += " Console.WriteLine(\"hello world\");\r\n";
sourceCode += " }\r\n";
sourceCode += "}\r\n";
CompilerResults results = codeProvider.CompileAssemblyFromSource(parameters, sourceCode);
Assembly assembly = null;
if (!results.Errors.HasErrors)
{
assembly = results.CompiledAssembly;
}
}
Update 1: This question may be related: Dynamically loading and unloading a a dll generated using CSharpCodeProvider
Update 2: Trying to understand application domains more, I found this: What is an application domain - an explanation for .Net beginners
Update 3: To clarify, I'm looking for a solution that provides the same functionality as the code above (compiling and providing access to generated code) without leaking memory. It looks like the solution will involve creating a new AppDomain and marshaling.
I think I have a working solution. Thanks to everyone for pointing me in the right direction (I hope).
Assemblies can't be unloaded directly, but AppDomains can. I created a helper library that gets loaded in a new AppDomain and is able to compile a new assembly from code. Here's what the class in that helper library looks like:
public class CompilerRunner : MarshalByRefObject
{
private Assembly assembly = null;
public void PrintDomain()
{
Console.WriteLine("Object is executing in AppDomain \"{0}\"",
AppDomain.CurrentDomain.FriendlyName);
}
public bool Compile(string code)
{
CSharpCodeProvider codeProvider = new CSharpCodeProvider();
CompilerParameters parameters = new CompilerParameters();
parameters.GenerateInMemory = true;
parameters.GenerateExecutable = false;
parameters.ReferencedAssemblies.Add("system.dll");
CompilerResults results = codeProvider.CompileAssemblyFromSource(parameters, code);
if (!results.Errors.HasErrors)
{
this.assembly = results.CompiledAssembly;
}
else
{
this.assembly = null;
}
return this.assembly != null;
}
public object Run(string typeName, string methodName, object[] args)
{
Type type = this.assembly.GetType(typeName);
return type.InvokeMember(methodName, BindingFlags.InvokeMethod, null, assembly, args);
}
}
It's very basic, but was enough for testing. PrintDomain is there to verify that it does live in my new AppDomain. Compile takes some source code and tries to create an assembly. Run lets us test executing static methods from the given source code.
Here's how I use the helper library:
static void CreateCompileAndRun()
{
AppDomain domain = AppDomain.CreateDomain("MyDomain");
CompilerRunner cr = (CompilerRunner)domain.CreateInstanceFromAndUnwrap("CompilerRunner.dll", "AppDomainCompiler.CompilerRunner");
cr.Compile("public class Hello { public static string Say() { return \"hello\"; } }");
string result = (string)cr.Run("Hello", "Say", new object[0]);
AppDomain.Unload(domain);
}
It basically creates the domain, creates an instance of my helper class (CompilerRunner), uses it to compile a new assembly (hidden), runs some code from that new assembly, and then unloads the domain to free up memory.
You'll notice the use of MarshalByRefObject and CreateInstanceFromAndUnwrap. These are important for ensuring that the helper library really does live in the new domain.
If anyone notices any problems or has suggestions for improving this, I'd love to hear them.
Unloading an assembly is not supported. Some information on why can be found here.
Some information on using an AppDomain can be found here.
You may also find this blog entry useful: Using AppDomain to Load and Unload Dynamic Assemblies. It provides some example code demonstrating how create an AppDomain, load a (dynamic) assembly into it, do some work in the new AppDomain then unload it.
Edit: fixed link as pointed out in comments below.
Can you wait until .NET 4.0? With it you can use expression trees and the DLR to dynamically generate code without the code-gen memory loss issue.
Another option is to use .NET 3.5 with a dynamic language like IronPython.
EDIT: Expression Tree Example
http://www.infoq.com/articles/expression-compiler