Suppose my code possesses the knowledge about the metadata of a
nonexistent class library "mytest.dll", such as the types in this library, the functions of the types, the parameters and return types of the functions, etc.
How does my code manufacture this DLL using techniques such as reflection?
I know my code can generate the "mytest.cs" text file, then execute the compiler to produce the DLL, then delete the "mytest.cs" file. Just want to know if there are "more advanced" or "cooler" ways to do it.
Thanks.
There are 4 main steps in the process to compile and execute dynamic .net scripts from your application, even really complex scenarios can be simplified in this way:
Generate the code
Compile the script
Load the assembly
Execute the code
Lets generate a simple Hello Generated C# World App right now!:
Create a method that will generate an assembly that has 1 class called HelloWorldApp, this class has 1 method called GenerateMessage it will have X input parameters that will be integers, it will return a CSV string of the arguments that were passed in to it.
This solution requires the following package to be installed:
PM> Install-Package 'Microsoft.CodeAnalysis.CSharp.Scripting'
And will require the following using statements:
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.Emit;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
Orchestration
The following method encapsulates the above steps:
private static void GenerateAndExecuteApp(int numberOfParameters)
{
string nameSpace = "Dynamic.Example";
string className = "HelloWorldApp";
string methodName = "GenerateMessage";
// 1. Generate the code
string script = BuildScript(nameSpace, className, methodName, numberOfParameters);
// 2. Compile the script
// 3. Load the Assembly
Assembly dynamicAssembly = CompileScript(script);
// 4. Execute the code
int[] arguments = Enumerable.Range(1, numberOfParameters).ToArray();
string message = ExecuteScript(dynamicAssembly, nameSpace, className, methodName, arguments);
Console.Out.WriteLine(message);
}
Generate the code
You say you already have item 1 sorted out, you can use StringBuilder, T4 templates or other mechanisms to generate the code files.
generating the code itself is its own question if you need help with that.
However, for our demo app, the following would work:
private static string BuildScript(string nameSpace, string className, string methodName, int numberOfParameters)
{
StringBuilder code = new StringBuilder();
code.AppendLine("using System;");
code.AppendLine("using System.Linq;");
code.AppendLine();
code.AppendLine($"namespace {nameSpace}");
code.AppendLine("{");
code.AppendLine($" public class {className}");
code.AppendLine(" {");
var parameterNames = Enumerable.Range(0, numberOfParameters).Select(x => $"p{x}").ToList();
code.Append($" public string {methodName}(");
code.Append(String.Join(",", parameterNames.Select(x => $"int {x}")));
code.AppendLine(")");
code.AppendLine(" {");
code.Append(" return $\"");
code.Append(String.Join(",", parameterNames.Select(x => $"{x}={{{x}}}")));
code.AppendLine("\";");
code.AppendLine(" }");
code.AppendLine(" }");
code.AppendLine("}");
return code.ToString();
}
For an input value of 3, the following code is generated:
using System;
using System.Linq;
namespace Dynamic.Example
{
public class HelloWorldApp
{
public string GenerateMessage(int p0,int p1,int p2)
{
return $"p0={p0},p1={p1},p2={p2}";
}
}
}
Compile the script (and Load it)
These are two discrete steps, however it is easiest to code them together in the same method, for this example we will ignore the generated dll and load the assembly directly into memory, that is generally the more likely use case for this type of scripting scenario anyway.
The hardest element of this is usually the referencing of the relevant dlls. There are a number of ways to achieve this, including loading all the dlls that are in the current executing context, I find a simple way to do this is to access the Assembly reference from the Type reference for the types we want to use inside the dynamic script:
List<string> dlls = new List<string> {
typeof(object).Assembly.Location,
typeof(Enumerable).Assembly.Location
};
Cut a long story short, this method compiles and loads the assembly into memory. It includes some crude compilation error handling, just to demonstrate how to do it:
private static Assembly CompileScript(string script)
{
SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(script);
// use "mytest.dll" if you want, random works well enough
string assemblyName = System.IO.Path.GetRandomFileName();
List<string> dlls = new List<string> {
typeof(object).Assembly.Location,
typeof(Enumerable).Assembly.Location
};
MetadataReference[] references = dlls.Distinct().Select(x => MetadataReference.CreateFromFile(x)).ToArray();
CSharpCompilation compilation = CSharpCompilation.Create(
assemblyName,
syntaxTrees: new[] { syntaxTree },
references: references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
// Now we actually compile the script, this includes some very crude error handling, just to show you can
using (var ms = new MemoryStream())
{
EmitResult result = compilation.Emit(ms);
if (!result.Success)
{
IEnumerable<Diagnostic> failures = result.Diagnostics.Where(diagnostic =>
diagnostic.IsWarningAsError ||
diagnostic.Severity == DiagnosticSeverity.Error);
List<string> errors = new List<string>();
foreach (Diagnostic diagnostic in failures)
{
//errors.AddDistinct(String.Format("{0} : {1}", diagnostic.Id, diagnostic.Location, diagnostic.GetMessage()));
errors.Add(diagnostic.ToString());
}
throw new ApplicationException("Compilation Errors: " + String.Join(Environment.NewLine, errors));
}
else
{
ms.Seek(0, SeekOrigin.Begin);
return Assembly.Load(ms.ToArray());
}
}
}
Execute the code
Finally, we can use reflection to instantiate an instance of the new app and then we can obtain a reference to the method and it. The name of the parameters is irrelevant, as long
we pass them through in the correct order:
for this demo the order is sort of irrelevant to, given they are all the same type ;)
private static string ExecuteScript(Assembly assembly, string nameSpace, string className, string methodName, int[] arguments)
{
var appType = assembly.GetType($"{nameSpace}.{className}");
object app = Activator.CreateInstance(appType);
MethodInfo method = appType.GetMethod(methodName);
object result = method.Invoke(app, arguments.Cast<object>().ToArray());
return result as string;
}
Output
The final output from all this for our method with 3 passed into it is:
p0=1,p1=2,p2=3
So that was super crude, you can bypass most of the indirect reflection aspects through the use of Interfaces. If your generated script inherits from types or interfaces that the calling code also has a strong reference to, then ExecuteScript in the above example might look like this:
private static string ExecuteScript(Assembly assembly, string nameSpace, string className)
{
var appType = assembly.GetType($"{nameSpace}.{className}");
object app = Activator.CreateInstance(appType);
if (app is KnownInterface known)
{
return known.GenerateMessage(1,2,3);
}
throw new NotSupportedException("Couldn't resolve known type");
}
The major benefit to using an interface or base class reference is that you can natively set properties or call other methods without having to reflect references to them all or to resort to using dynamic which would work, but becomes a bit harder to debug.
Of course the interface solution is hard to implement when we had a variable number of parameters, so that's not the best example, usually with dynamic scripts you would construct a known environment, say a known class and methods, but you might want to inject custom code into the body of the method.
It's a bit of fun in the end, but this simple example shows that C# can be used as a runtime scripting engine without too much trouble.
Related
I writed a SourceGenerator, but how do I test it?
Main issue is how to imitate GeneratorExecutionContext (or just Compilation inside it) which generator gets into Execute method. I think there is a proper way to make fake SyntaxTrees for unit testing, but I cant find it. There are many articles about source generators itself, but none of them explain how to test generators.
You should look at official Source Generators Cookbook
There is example from it:
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
namespace GeneratorTests.Tests
{
[TestClass]
public class GeneratorTests
{
[TestMethod]
public void SimpleGeneratorTest()
{
// Create the 'input' compilation that the generator will act on
Compilation inputCompilation = CreateCompilation(#"
namespace MyCode
{
public class Program
{
public static void Main(string[] args)
{
}
}
}
");
// directly create an instance of the generator
// (Note: in the compiler this is loaded from an assembly, and created via reflection at runtime)
CustomGenerator generator = new CustomGenerator();
// Create the driver that will control the generation, passing in our generator
GeneratorDriver driver = CSharpGeneratorDriver.Create(generator);
// Run the generation pass
// (Note: the generator driver itself is immutable, and all calls return an updated version of the driver that you should use for subsequent calls)
driver = driver.RunGeneratorsAndUpdateCompilation(inputCompilation, out var outputCompilation, out var diagnostics);
// We can now assert things about the resulting compilation:
Debug.Assert(diagnostics.IsEmpty); // there were no diagnostics created by the generators
Debug.Assert(outputCompilation.SyntaxTrees.Count() == 2); // we have two syntax trees, the original 'user' provided one, and the one added by the generator
Debug.Assert(outputCompilation.GetDiagnostics().IsEmpty); // verify the compilation with the added source has no diagnostics
// Or we can look at the results directly:
GeneratorDriverRunResult runResult = driver.GetRunResult();
// The runResult contains the combined results of all generators passed to the driver
Debug.Assert(runResult.GeneratedTrees.Length == 1);
Debug.Assert(runResult.Diagnostics.IsEmpty);
// Or you can access the individual results on a by-generator basis
GeneratorRunResult generatorResult = runResult.Results[0];
Debug.Assert(generatorResult.Generator == generator);
Debug.Assert(generatorResult.Diagnostics.IsEmpty);
Debug.Assert(generatorResult.GeneratedSources.Length == 1);
Debug.Assert(generatorResult.Exception is null);
}
private static Compilation CreateCompilation(string source)
=> CSharpCompilation.Create("compilation",
new[] { CSharpSyntaxTree.ParseText(source) },
new[] { MetadataReference.CreateFromFile(typeof(Binder).GetTypeInfo().Assembly.Location) },
new CSharpCompilationOptions(OutputKind.ConsoleApplication));
}
}
In addition to the Source Generators Cookbook mentioned in the other answer:
The cookbook solution allows you to generate some code and then compare your results to expected, also check for warnings and compilation exceptions etc.
Now, you can additionally EXECUTE the generated code to make sure it's running correctly. For that change the project reference in the test-project like this:
<ProjectReference Include="..\MyGenerator\MyGenerator.csproj"
ReferenceOutputAssembly="true"
OutputItemType="Analyzer" />
And then simply call the generated code from your unit tests, like you would in the consumer project.
I am trying to directly invoke IronPython's built-in modules from C#. It looks like I'm missing some important initialization, that I can't find anywhere in the code.
Here's what I do:
namespace py.consoleio
{
using IronPython.Runtime;
using Microsoft.Scripting.Hosting;
using Microsoft.Scripting.Hosting.Providers;
using Microsoft.Scripting.Runtime;
public static class consoleio
{
public static string name;
static void Main()
{
var setup = new ScriptRuntimeSetup();
setup.LanguageSetups.Add(
IronPython.Hosting.Python.CreateLanguageSetup(null));
var dlrRuntime = new ScriptRuntime(setup);
var scriptDomainManager = HostingHelpers.GetDomainManager(dlrRuntime);
var pythonContext = new PythonContext(scriptDomainManager, null);
var context = new CodeContext(new PythonDictionary(), new ModuleContext(new PythonDictionary(), DefaultContext.DefaultPythonContext));
name = IronPython.Modules.Builtin.input(context, "What is your name?\n");
IronPython.Modules.Builtin.print(context, "Hi, %s.", consoleio.name);
System.GC.KeepAlive(pythonContext);
}
}
}
That properly outputs "What is your name?", but then crashes trying to decode input: unknown encoding: cp437.
Now I've already found, that encodings are initialized in Src/StdLib/Lib/encodings/init.py
I can't find how it gets to loading this module in a normal IronPython run (e.g. a console host), so I can't reproduce it in C# program.
My goal here is to invoke IronPython functions without dynamic dispatch.
UPD. Now I also tried to do this:
var engine = Python.CreateEngine();
this.ScriptDomainManager = HostingHelpers.GetDomainManager(engine.Runtime);
to the same result
Figured that one out: encodings module is implemented in Python in IronPython (core modules are in C#). It always worked with IronPythonConsole project, because it implicitly adds IronPython source for standard libraries to Python path. I just had to explicitly specify path like this:
var options = new Dictionary<string, object> { ["SearchPaths"] = path };
var engine = Python.CreateEngine(options);
I have simple IScript interface. And I want enforce that all scripts implement it.
public interface IScript<T>
{
T Execute(object[] args);
}
I want to use Roslyn scripting API to achive this. Something like this is possible with CSScript (see Interface Alignment).
var code = #"
using System;
using My.Namespace.With.IScript;
public class Script : IScript<string>
{
public string Execute()
{
return ""Hello from script!"";
}
}
";
var script = CSharpScript.Create(code, ScriptOptions.Default); // + Load all assemblies and references
script.WithInterface(typeof(IScript<string>)); // I need something like this, to enforce interface
script.Compile();
string result = script.Execute(); // and then execute script
Console.WriteLine(result); // print "Hello from script!"
Type safety is a static thing enforced a compile time (of your application). Creating and running a CSharpScript is done at runtime. So you cannot enforce type safety at runtime.
Maybe CSharpScript is not the right way to go. By using this SO answer,
You can compile a piece of C# code into memory and generate assembly bytes with Roslyn.
You would then change the line
object obj = Activator.CreateInstance(type);
to
IScript<string> obj = Activator.CreateInstance(type) as IScript<string>;
if (obj != null) {
obj.Execute(args);
}
I have a webdriver solution that has 10 or so projects in it. 1 core assembly/dll that holds all the common methods and 9 other test assemblies that use those methods in their tests.
I need to access an embedded resource for one of those 9 assemblies but I need to do it from inside the core dll. What's the best way to do that.
namespace = webdriver.core
json.cs - reads a json file and returns it as a string
namespace = webdriver.marketplacestest
marketplace1Test.cs - calls one of the methods in json.cs such as getName();
profile.json - holds {"marketplace1" : "Amazon"}
calling an embedded resource from a known namespace is easy. I did that like this:
private static string fromEmbeddedResource(string myNamespace, string myFolder, string fileName)
{
string result;
using (Stream stream = Assembly.GetExecutingAssembly().GetManifestResourceStream(myNamespace + "." + myFolder + "." + fileName))
using (StreamReader reader = new StreamReader(stream))
{
result = reader.ReadToEnd();
}
return result;
}
As you can see, I just call the following and I have the file as a string
string json = fromEmbeddedResource("WebDriver.Core", "CFG", "pid.json");
It's harder though when the file is embedded in one of my test dlls. Anyone know how I can access or get the assembly's namespace? I tried...
Assembly.GetCallingAssembly().GetTypes();
but it looks like it's pulling types from the WebDriver.Core.dll assembly and not the WebDriver.Marketplace1.dll assembly.
I was able to figure it out. The problem I had was the calling assembly wasn't the correct assembly because I was calling a method in my core dll that called another method in my core dll. I got it working by passing the assembly along but that was expensive. To make things more efficient I modified my static SettingsRepository class that holds the two assemblies in a dictionary. That way I can pass in a string of "core" or "test" and pull the assembly without having to determine if I'm using an executing assembly or calling assembly each time.
private static Dictionary<string, object> _assembly = new Dictionary<string,object>();
public static Assembly getAssembly (string type)
{
return _assembly[type] as Assembly;
}
public static void addAssembly(string myType, Assembly assembly)
{
bool containsKey = _assembly.ContainsKey(myType);
if (!containsKey)
{
_assembly.Add(myType, assembly);
}
}
When I start a test, I always initialize my driver class first so i added the following two sets to that constructor.
Settings.addAssembly("core", Assembly.GetExecutingAssembly());
Settings.addAssembly("test", Assembly.GetCallingAssembly());
Now I can call either assembly I want anytime I want and know which one I'm getting.
I asked a while ago how to restrict plugins access ( I want to prevent them from writing to the disk or network ) and i was told to use AppDomain. I have searched and tried and failed on how to get this working.
Can anyone provide some information so i can get started, simply put make a AppDomain that does not allows writing to the file or network.
For .net framework 4.0, please follow the following code from this MSDN article.
The following example implements the procedure in the previous section. In the example, a project named Sandboxer in a Visual Studio solution also contains a project named UntrustedCode, which implements the class UntrustedClass. This scenario assumes that you have downloaded a library assembly containing a method that is expected to return true or false to indicate whether the number you provided is a Fibonacci number. Instead, the method attempts to read a file from your computer. The following example shows the untrusted code.
using System;
using System.IO;
namespace UntrustedCode
{
public class UntrustedClass
{
// Pretend to be a method checking if a number is a Fibonacci
// but which actually attempts to read a file.
public static bool IsFibonacci(int number)
{
File.ReadAllText("C:\\Temp\\file.txt");
return false;
}
}
}
The following example shows the Sandboxer application code that executes the untrusted code.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Security;
using System.Security.Policy;
using System.Security.Permissions;
using System.Reflection;
using System.Runtime.Remoting;
//The Sandboxer class needs to derive from MarshalByRefObject so that we can create it in another
// AppDomain and refer to it from the default AppDomain.
class Sandboxer : MarshalByRefObject
{
const string pathToUntrusted = #"..\..\..\UntrustedCode\bin\Debug";
const string untrustedAssembly = "UntrustedCode";
const string untrustedClass = "UntrustedCode.UntrustedClass";
const string entryPoint = "IsFibonacci";
private static Object[] parameters = { 45 };
static void Main()
{
//Setting the AppDomainSetup. It is very important to set the ApplicationBase to a folder
//other than the one in which the sandboxer resides.
AppDomainSetup adSetup = new AppDomainSetup();
adSetup.ApplicationBase = Path.GetFullPath(pathToUntrusted);
//Setting the permissions for the AppDomain. We give the permission to execute and to
//read/discover the location where the untrusted code is loaded.
PermissionSet permSet = new PermissionSet(PermissionState.None);
permSet.AddPermission(new SecurityPermission(SecurityPermissionFlag.Execution));
//We want the sandboxer assembly's strong name, so that we can add it to the full trust list.
StrongName fullTrustAssembly = typeof(Sandboxer).Assembly.Evidence.GetHostEvidence<StrongName>();
//Now we have everything we need to create the AppDomain, so let's create it.
AppDomain newDomain = AppDomain.CreateDomain("Sandbox", null, adSetup, permSet, fullTrustAssembly);
//Use CreateInstanceFrom to load an instance of the Sandboxer class into the
//new AppDomain.
ObjectHandle handle = Activator.CreateInstanceFrom(
newDomain, typeof(Sandboxer).Assembly.ManifestModule.FullyQualifiedName,
typeof(Sandboxer).FullName
);
//Unwrap the new domain instance into a reference in this domain and use it to execute the
//untrusted code.
Sandboxer newDomainInstance = (Sandboxer) handle.Unwrap();
newDomainInstance.ExecuteUntrustedCode(untrustedAssembly, untrustedClass, entryPoint, parameters);
}
public void ExecuteUntrustedCode(string assemblyName, string typeName, string entryPoint, Object[] parameters)
{
//Load the MethodInfo for a method in the new Assembly. This might be a method you know, or
//you can use Assembly.EntryPoint to get to the main function in an executable.
MethodInfo target = Assembly.Load(assemblyName).GetType(typeName).GetMethod(entryPoint);
try
{
//Now invoke the method.
bool retVal = (bool)target.Invoke(null, parameters);
}
catch (Exception ex)
{
// When we print informations from a SecurityException extra information can be printed if we are
//calling it with a full-trust stack.
(new PermissionSet(PermissionState.Unrestricted)).Assert();
Console.WriteLine("SecurityException caught:\n{0}", ex.ToString());
CodeAccessPermission.RevertAssert();
Console.ReadLine();
}
}
}
I guess this is what you need, if I understand correctly your point.
System.Security.PermissionSet ps =
new System.Security.PermissionSet(System.Security.Permissions.PermissionState.None);
ps.AddPermission(new System.Security.Permissions.FileIOPermission(System.Security.Permissions.FileIOPermissionAccess.NoAccess, "C:\\"));
System.Security.Policy.PolicyLevel pl = System.Security.Policy.PolicyLevel.CreateAppDomainLevel();
pl.RootCodeGroup.PolicyStatement = new System.Security.Policy.PolicyStatement(ps);
AppDomain.CurrentDomain.SetAppDomainPolicy(pl);
System.Reflection.Assembly myPluginAssembly = AppDomain.CurrentDomain.Load("MyPluginAssembly");
Is this more precisely what you meant?
Notice that you may provide an array of string containg the paths where you don't want the plugin to have access. You may provide if when initializing the new instance of FileIOPermission class.
Let me know if this helps. :-)
If you're using plugins, you might perhaps know about proxies.
While loading your assembly through a proxy, you can specify the security policy level for this particular assembly through the LoadAssembly() method or so, if I remember correctly. In other words, this is done through reflection.
I know my answer isn't that much detailed, but I hope it will give you an idea of where to look for your solution. I shall take an eye out to find further details on the subject so that I may be of better help. =)
Hope you will share your findings when you've done it.