Develop Reference

Cannot get SyntaxTree from Compilation object

I'm a beginner of roslyn, so I tried to start learning it by making a very simple console application, which is introduced in the famous tutorial site. (https://riptutorial.com/roslyn/example/16545/introspective-analysis-of-an-analyzer-in-csharp), and it didn't work well.
The Cosole Application I made is of .NET Framework (target Framework version is 4.7.2), and not of .NET Core nor .NET standard.
I added the NuGet package Microsoft.CodeAnalysis, and Microsoft.CodeAnalysis.Workspaces.MSBuild, then wrote a simple code as I show below.
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.MSBuild;
using System;
using System.Linq;
namespace SimpleRoslynConsole
{
class Program
{
static void Main(string[] args)
{
// Declaring a variable with the current project file path.
// *** You have to change this path to fit your development environment.
const string projectPath =
#"C:\Users\[MyName]\Source\Repos\RoslynTrialConsole01\RoslynTrialConsole01.csproj";
var workspace = MSBuildWorkspace.Create();
var project = workspace.OpenProjectAsync(projectPath).Result;
// [**1]Getting the compilation.
var compilation = project.GetCompilationAsync().Result;
// [**2]As this is a simple single file program, the first syntax tree will be the current file.
var syntaxTree = compilation.SyntaxTrees.FirstOrDefault();
if (syntaxTree != null)
{
var rootSyntaxNode = syntaxTree.GetRootAsync().Result;
var firstLocalVariablesDeclaration = rootSyntaxNode.DescendantNodesAndSelf()
.OfType<LocalDeclarationStatementSyntax>().First();
var firstVariable = firstLocalVariablesDeclaration.Declaration.Variables.First();
var variableInitializer = firstVariable.Initializer.Value.GetFirstToken().ValueText;
Console.WriteLine(variableInitializer);
}
else
{
Console.WriteLine("Could not get SyntaxTrees from this projects.");
}
Console.WriteLine("Hit any key.");
Console.ReadKey();
}
}
}
My problem is that, SyntaxTrees property of Compilation object returns null in [**2]mark. Naturally, following FirstOrDefault method returns null.
I've tried several other code. I found I could get SyntaxTree from CSharp code text, by using CSharpSyntaxTree.ParseText method. But I couldn't get any from source code, by the sequence of
var workspace = MSBuildWorkspace.Create();
var project = workspace.OpenProjectAsync(projectPath).Result;
var compilation = project.GetCompilationAsync().Result;
What I'd like to know is if I miss something to get Syntax information from source code by using above process.
I'll appreciate someone give me a good advice.

I think the issue is that .net framework projects have their source files paths within their .csproj. And opening project works right away.
For .net core project you have no such information and, maybe, this is why Workspace instance doesn't know what to load and so loads nothing.
At least specifying .cs files as added documents does the trick. Try to apply this:
static class ProjectExtensions
{
public static Project AddDocuments(this Project project, IEnumerable<string> files)
{
foreach (string file in files)
{
project = project.AddDocument(file, File.ReadAllText(file)).Project;
}
return project;
}
private static IEnumerable<string> GetAllSourceFiles(string directoryPath)
{
var res = Directory.GetFiles(directoryPath, "*.cs", SearchOption.AllDirectories);
return res;
}
public static Project WithAllSourceFiles(this Project project)
{
string projectDirectory = Directory.GetParent(project.FilePath).FullName;
var files = GetAllSourceFiles(projectDirectory);
var newProject = project.AddDocuments(files);
return newProject;
}
}
Method WithAllsourceFiles will return you the project, compilation of which will in its turn have all syntax trees you would expect of it, as you would have in Visual Studio

MsBuildWorkspace won't work correctly unless you have all the same redirects in your app's app.config file that msbuild.exe.config has in it. Without the redirects, it's probably failing to load the msbuild libraries. You need to find the msbuild.exe.config file that is on your system and copy the <assemblyBinding> elements related to Microsoft.Build assemblies into your app.config. Make sure you place them under the correct elements configuration/runtime.

I searched various sample programs on the net and found the most reliable and safest method. The solution is to create a static method which returns SyntaxTrees in designated File as follow.
private static Compilation CreateTestCompilation()
{
var found = false;
var di = new DirectoryInfo(Environment.CurrentDirectory);
var fi = di.GetFiles().Where((crt) => { return crt.Name.Equals("program.cs", StringComparison.CurrentCultureIgnoreCase); }).FirstOrDefault();
while ((fi == null) || (di.Parent == null))
{
di = new DirectoryInfo(di.Parent.FullName);
fi = di.GetFiles().Where((crt) => { return crt.Name.Equals("program.cs", StringComparison.CurrentCultureIgnoreCase); }).FirstOrDefault();
if (fi != null)
{
found = true;
break;
}
}
if (!found)
{
return null;
}
var targetPath = di.FullName + #"\Program.cs";
var targetText = File.ReadAllText(targetPath);
var targetTree =
CSharpSyntaxTree.ParseText(targetText)
.WithFilePath(targetPath);
var target2Path = di.FullName + #"\TypeInferenceRewriter.cs";
var target2Text = File.ReadAllText(target2Path);
var target2Tree =
CSharpSyntaxTree.ParseText(target2Text)
.WithFilePath(target2Path);
SyntaxTree[] sourceTrees = { programTree, target2Tree };
MetadataReference mscorlib =
MetadataReference.CreateFromFile(typeof(object).Assembly.Location);
MetadataReference codeAnalysis =
MetadataReference.CreateFromFile(typeof(SyntaxTree).Assembly.Location);
MetadataReference csharpCodeAnalysis =
MetadataReference.CreateFromFile(typeof(CSharpSyntaxTree).Assembly.Location);
MetadataReference[] references = { mscorlib, codeAnalysis, csharpCodeAnalysis };
return CSharpCompilation.Create("TransformationCS",
sourceTrees,
references,
new CSharpCompilationOptions(
OutputKind.ConsoleApplication));
}
And the caller program will be like this.
static void Main(string[] args)
{
var test = CreateTestCompilation();
if (test == null)
{
return;
}
foreach (SyntaxTree sourceTree in test.SyntaxTrees)
{
Console.WriteLine(souceTree.ToFullString());
}
}
Of course, many improvements are needed to put it to practical use.

Roslyn in memory compilation: CS0103: The name 'Console' does not exist in the current context

So i'm having this problem.
I'm trying to compile code in memory and adding namespace references by searching the syntax tree so i do not add them manually. Trying to simulate how Visual Studio maybe does it.
I'm a bit over my head in the compilation department. Even if i add a metadata reference to System while reading the syntax tree it does not find System.Console.
The key is that i want it to include the assemblies by itself, i do not want to add a "MetadataReference.CreateFromFile(....,"System.Console").
I explained the code below so that is clear what is happening.
class App
{
static void Main(string[] args)
{
//creating the syntax tree for the program
SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(#"
namespace ns{
using System;
public class App{
public static void Main(string[] args){
Console.Write(""dada"");
}
}
}");
//creating options that tell the compiler to output a console application
var options = new CSharpCompilationOptions(
OutputKind.ConsoleApplication,
optimizationLevel: OptimizationLevel.Debug,
allowUnsafe: true);
//creating the compilation
var compilation = CSharpCompilation.Create(Path.GetRandomFileName(), options: options);
//adding the syntax tree
compilation = compilation.AddSyntaxTrees(syntaxTree);
//getting the local path of the assemblies
var assemblyPath = Path.GetDirectoryName(typeof(object).Assembly.Location);
List<MetadataReference> references = new List<MetadataReference>();
//adding the core dll containing object and other classes
references.Add(MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.Private.CoreLib.dll")));
references.Add(MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "mscorlib.dll")));
//gathering all using directives in the compilation
var usings = compilation.SyntaxTrees.Select(tree => tree.GetRoot().ChildNodes().OfType<UsingDirectiveSyntax>()).SelectMany(s => s).ToArray();
//for each using directive add a metadatareference to it
foreach (var u in usings)
{
references.Add(MetadataReference.CreateFromFile(Path.Combine(assemblyPath, u.Name.ToString() + ".dll")));
}
//add the reference list to the compilation
compilation=compilation.AddReferences(references);
//compile
using (var ms = new MemoryStream())
{
EmitResult result = compilation.Emit(ms);
if (!result.Success)
{
IEnumerable<Diagnostic> failures = result.Diagnostics.Where(diagnostic =>
diagnostic.IsWarningAsError ||
diagnostic.Severity == DiagnosticSeverity.Error);
foreach (Diagnostic diagnostic in failures)
{
Console.Error.WriteLine("{0}: {1}, {2}", diagnostic.Id, diagnostic.GetMessage(), diagnostic.Location);
}
}
else
{
ms.Seek(0, SeekOrigin.Begin);
AssemblyLoadContext context = AssemblyLoadContext.Default;
Assembly assembly = context.LoadFromStream(ms);
assembly.EntryPoint.Invoke(null, new object[] { new string[] { "arg1", "arg2", "etc" } });
}
}
}
}

In the .net core System.Console lives in the System.Console.dll. So you need to add reference on it
You need to add reference on the System.Runtime.dll to correctly resolve the predefined types: object, bool and so on
SyntaxNode.ChildNodes() returns only child, that means it doesn't return the descendents nodes, so if you want to get all UsingDirectiveSyntax you should change your logic. As one of way just use SyntaxNode.DescendantNodes()
After applying all suggestions you just get something likes this (The parts that didn't change will skipped):
...
//adding the core dll containing object and other classes
references.Add(MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.Private.CoreLib.dll")));
references.Add(MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.Console.dll")));
references.Add(MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.Runtime.dll")));
//gathering all using directives in the compilation
var usings = compilation.SyntaxTrees.Select(tree => tree.GetRoot().DescendantNodes().OfType<UsingDirectiveSyntax>()).SelectMany(s => s).ToArray();
...

How to debug dll generated from Roslyn compilation?

I’m using Roslyn CSharpCompilation to generate dll files for my plugins – files have OptimizationLevel.Debug and pdb file is generated. Next I’m loading those files to my program (UWP + .NET Standard 2.0 libs) using Assembly.Load and create instance of types I’m interested in. My problem is that I can’t get Visual Studio (version 2017 15.7.3) to find source code when I’m debugging – it is treading it like external library, so when exception is thrown inside I can't find where. I have tired to search solution on stackoverflow but all solution are not working. I have checked this:
Pdb is generated
Module window in VS shows that symbols are loaded
Tried different version of Assembly Load/LoadFrom
Setting “Use
Managed Compatibility Mode” in debug options
Is there any way to make the file debuggable? Maybe I have to use some roslyn option while compiling or change something in VS?

The code sample below should help you on your way. It`s based on the code generation part of thlamare IOC container lamar, the successor of StructureMap made by Jeremy D Miller.
I have only added debugging capabilities. The trick was to make the source text embeddable, choosing the right formats, and setting encoding values where needed.
Check out the original work for more details of e.g. adding references.
public Assembly CreateAssembly(string code)
{
var encoding = Encoding.UTF8;
var assemblyName = Path.GetRandomFileName();
var symbolsName = Path.ChangeExtension(assemblyName, "pdb");
var sourceCodePath = "generated.cs";
var buffer = encoding.GetBytes(code);
var sourceText = SourceText.From(buffer, buffer.Length, encoding, canBeEmbedded: true);
var syntaxTree = CSharpSyntaxTree.ParseText(
sourceText,
new CSharpParseOptions(),
path: sourceCodePath);
var syntaxRootNode = syntaxTree.GetRoot() as CSharpSyntaxNode;
var encoded = CSharpSyntaxTree.Create(syntaxRootNode, null, sourceCodePath, encoding);
var optimizationLevel = OptimizationLevel.Debug;
CSharpCompilation compilation = CSharpCompilation.Create(
assemblyName,
syntaxTrees: new[] { encoded },
references: references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)
.WithOptimizationLevel(optimizationLevel)
.WithPlatform(Platform.AnyCpu)
);
using (var assemblyStream = new MemoryStream())
using (var symbolsStream = new MemoryStream())
{
var emitOptions = new EmitOptions(
debugInformationFormat: DebugInformationFormat.PortablePdb,
pdbFilePath: symbolsName);
var embeddedTexts = new List<EmbeddedText>
{
EmbeddedText.FromSource(sourceCodePath, sourceText),
};
EmitResult result = compilation.Emit(
peStream: assemblyStream,
pdbStream: symbolsStream,
embeddedTexts: embeddedTexts,
options: emitOptions);
if (!result.Success)
{
var errors = new List<string>();
IEnumerable<Diagnostic> failures = result.Diagnostics.Where(diagnostic =>
diagnostic.IsWarningAsError ||
diagnostic.Severity == DiagnosticSeverity.Error);
foreach (Diagnostic diagnostic in failures)
errors.Add($"{diagnostic.Id}: {diagnostic.GetMessage()}");
throw new Exception(String.Join("\n", errors));
}
Console.WriteLine(code);
assemblyStream.Seek(0, SeekOrigin.Begin);
symbolsStream?.Seek(0, SeekOrigin.Begin);
var assembly = AssemblyLoadContext.Default.LoadFromStream(assemblyStream, symbolsStream);
return assembly;
}
}
Usage:
[Test]
public void Verify()
{
var code =
#"namespace Debuggable
{
public class HelloWorld
{
public string Greet(string name)
{
var result = ""Hello, "" + name;
return result;
}
}
}
";
var codeGenerator = new CodeGenerator();
var assembly = codeGenerator.CreateAssembly(code);
dynamic instance = assembly.CreateInstance("Debuggable.HelloWorld");
// Set breakpoint here
string result = instance.Greet("Roslyn");
result.Should().Be("Hello, Roslyn");
}

Compiling and running code at runtime in .NET Core 1.0

Is it possible to compile and run C# code at runtime in the new .NET Core (better .NET Standard Platform)?
I have seen some examples (.NET Framework), but they used NuGet packages that are not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0)

Option #1: Use the full C# compiler to compile an assembly, load it and then execute a method from it.
This requires the following packages as dependencies in your project.json:
"Microsoft.CodeAnalysis.CSharp": "1.3.0-beta1-20160429-01",
"System.Runtime.Loader": "4.0.0-rc2-24027",
Then you can use code like this:
var compilation = CSharpCompilation.Create("a")
.WithOptions(new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary))
.AddReferences(
MetadataReference.CreateFromFile(typeof(object).GetTypeInfo().Assembly.Location))
.AddSyntaxTrees(CSharpSyntaxTree.ParseText(
#"
using System;
public static class C
{
public static void M()
{
Console.WriteLine(""Hello Roslyn."");
}
}"));
var fileName = "a.dll";
compilation.Emit(fileName);
var a = AssemblyLoadContext.Default.LoadFromAssemblyPath(Path.GetFullPath(fileName));
a.GetType("C").GetMethod("M").Invoke(null, null);
Option #2: Use Roslyn Scripting. This will result in much simpler code, but it currently requires more setup:
Create NuGet.config to get packages from the Roslyn nightly feed:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="Roslyn Nightly" value="https://www.myget.org/F/roslyn-nightly/api/v3/index.json" />
</packageSources>
</configuration>
Add the following package as a dependency to project.json (notice that this is package from today. You will need different version in the future):
"Microsoft.CodeAnalysis.CSharp.Scripting": "1.3.0-beta1-20160530-01",
You also need to import dotnet (obsolete "Target Framework Moniker", which is nevertheless still used by Roslyn):
"frameworks": {
"netcoreapp1.0": {
"imports": "dotnet5.6"
}
}
Now you can finally use Scripting:
CSharpScript.EvaluateAsync(#"using System;Console.WriteLine(""Hello Roslyn."");").Wait();

I am just adding to svick's answer. If you want to keep the assembly in memory (rather than writing to a file) you can use the following method:
AssemblyLoadContext context = AssemblyLoadContext.Default;
Assembly assembly = context.LoadFromStream(ms);
This is different than in .NET 4.5.1 where the code is:
Assembly assembly = Assembly.Load(ms.ToArray());
My code targets both .NET 4.5.1 and .NET Standard, so I had to use directives to get around this problem. The full code example is here:
string code = CreateFunctionCode();
var syntaxTree = CSharpSyntaxTree.ParseText(code);
MetadataReference[] references = new MetadataReference[]
{
MetadataReference.CreateFromFile(typeof(object).GetTypeInfo().Assembly.Location),
MetadataReference.CreateFromFile(typeof(Hashtable).GetTypeInfo().Assembly.Location)
};
var compilation = CSharpCompilation.Create("Function.dll",
syntaxTrees: new[] { syntaxTree },
references: references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
StringBuilder message = new StringBuilder();
using (var ms = new MemoryStream())
{
EmitResult result = compilation.Emit(ms);
if (!result.Success)
{
IEnumerable<Diagnostic> failures = result.Diagnostics.Where(diagnostic =>
diagnostic.IsWarningAsError ||
diagnostic.Severity == DiagnosticSeverity.Error);
foreach (Diagnostic diagnostic in failures)
{
message.AppendFormat("{0}: {1}", diagnostic.Id, diagnostic.GetMessage());
}
return new ReturnValue<MethodInfo>(false, "The following compile errors were encountered: " + message.ToString(), null);
}
else
{
ms.Seek(0, SeekOrigin.Begin);
#if NET451
Assembly assembly = Assembly.Load(ms.ToArray());
#else
AssemblyLoadContext context = AssemblyLoadContext.Default;
Assembly assembly = context.LoadFromStream(ms);
#endif
Type mappingFunction = assembly.GetType("Program");
_functionMethod = mappingFunction.GetMethod("CustomFunction");
_resetMethod = mappingFunction.GetMethod("Reset");
}
}

Both previous answers didn't work for me in a .NET Core 2.2 environment on Windows. More references are needed.
So with the help of the https://stackoverflow.com/a/39260735/710069 solution, I have ended up with this code:
var dotnetCoreDirectory = System.Runtime.InteropServices.RuntimeEnvironment.GetRuntimeDirectory();
var compilation = CSharpCompilation.Create("LibraryName")
.WithOptions(new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary))
.AddReferences(
MetadataReference.CreateFromFile(typeof(object).GetTypeInfo().Assembly.Location),
MetadataReference.CreateFromFile(typeof(Console).GetTypeInfo().Assembly.Location),
MetadataReference.CreateFromFile(Path.Combine(dotnetCoreDirectory, "mscorlib.dll")),
MetadataReference.CreateFromFile(Path.Combine(dotnetCoreDirectory, "netstandard.dll")),
MetadataReference.CreateFromFile(Path.Combine(dotnetCoreDirectory, "System.Runtime.dll")))
.AddSyntaxTrees(CSharpSyntaxTree.ParseText(
#"public static class ClassName
{
public static void MethodName() => System.Console.WriteLine(""Hello C# Compilation."");
}"));
// Debug output. In case your environment is different it may show some messages.
foreach (var compilerMessage in compilation.GetDiagnostics())
Console.WriteLine(compilerMessage);
Than output library to file:
var fileName = "LibraryName.dll";
var emitResult = compilation.Emit(fileName);
if (emitResult.Success)
{
var assembly = AssemblyLoadContext.Default.LoadFromAssemblyPath(Path.GetFullPath(fileName));
assembly.GetType("ClassName").GetMethod("MethodName").Invoke(null, null);
}
or to memory stream:
using (var memoryStream = new MemoryStream())
{
var emitResult = compilation.Emit(memoryStream);
if (emitResult.Success)
{
memoryStream.Seek(0, SeekOrigin.Begin);
var context = AssemblyLoadContext.Default;
var assembly = context.LoadFromStream(memoryStream);
assembly.GetType("ClassName").GetMethod("MethodName").Invoke(null, null);
}
}

Why Mono.Cecil argues for method import while I have done it already?

Here is my code :
private void ModifyMethods()
{
SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(#"
using System;
namespace ToIL
{
public class Class1
{
public void Write()
{
Console.WriteLine(""Hello"");
}
}
}");
string assemblyName = System.IO.Path.GetRandomFileName();
MetadataReference[] references = new MetadataReference[]
{
MetadataReference.CreateFromFile(typeof(object).Assembly.Location),
MetadataReference.CreateFromFile(typeof(Enumerable).Assembly.Location)
};
CSharpCompilation compilation = CSharpCompilation.Create( assemblyName, syntaxTrees: new[] { syntaxTree }, references: references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
Mono.Cecil.AssemblyDefinition asm = null;
using (var ms = new MemoryStream())
{
var emitResult = compilation.Emit(ms);
if (emitResult.Success)
{
ms.Seek(0, SeekOrigin.Begin);
asm = Mono.Cecil.AssemblyDefinition.ReadAssembly(ms);
}
}
var class1 = asm.MainModule.Assembly.MainModule.Types.FirstOrDefault(T => T.Name == "Class1");
var Method1 = class1.Methods.FirstOrDefault(M => M.Name == "Write");
var ils = Method1.Body.Instructions;
System.Reflection.MethodInfo mWriteLine = typeof(Console).GetMethod("WriteLine", new Type[] { typeof(string) });
Mono.Cecil.AssemblyDefinition asmx = Mono.Cecil.AssemblyDefinition.ReadAssembly(#"EditAsm.exe");
var import = asmx.MainModule.Import(mWriteLine);
foreach (var type in asmx.MainModule.Types)
{
if (type.Name == "<Module>") continue;
foreach (var method in type.Methods)
{
var cilWorker = method.Body.GetILProcessor();
foreach (var il in ils) cilWorker.Append(il);
}
}
asmx.Write(#"d:\test.dll"); // Import Exception
}
What this code does is that compiles Write method inside Class1 of assembly ToIL. Then the IL(Instructions) of method body is stored in ils. Finally the instructions are added to every method of EditAsm.exe assembly.
As is provided I have imported WriteLine but still geting following exception at asmx.Write(#"d:\test.dll");
Member 'System.Void System.Console::WriteLine(System.String)' is declared in another module and needs to be imported

That's because your IL instructions belong to another module, so method references in them are invalid for the module you are adding them to. Import actually just creates a reference to external method (field, etc), but this reference is valid for specific module. All your IL instruction method call operands have references that belong to module which named "assemblyName" in your code. This line:
var import = asmx.MainModule.Import(mWriteLine);
just does nothing actually, because you don't use the return value. How you can use it (in general)? Like this:
cilWorker.Append(Instruction.Create(OpCodes.Call, import));
You see that creating method call instruction requires method reference for this specific module you are adding instruction to. Now, possible way to fix your problem:
foreach (var type in asmx.MainModule.Types) {
if (type.Name == "<Module>") continue;
foreach (var method in type.Methods) {
var cilWorker = method.Body.GetILProcessor();
foreach (var il in ils) {
// grab method reference
var methodRef = il.Operand as Mono.Cecil.MethodReference;
if (methodRef != null) {
// if it belongs to another module
if (methodRef.Module.Name == (assemblyName + ".dll")) {
// resolve it back to method definition and then import,
// now to the correct module. Assign result back to opcode operand
il.Operand = asmx.MainModule.Import(methodRef.Resolve());
}
}
cilWorker.Append(il);
}
}
}
Then your code will no longer throw.