Develop Reference

How to programmatically create a class library DLL using reflection?

Suppose my code possesses the knowledge about the metadata of a
nonexistent class library "mytest.dll", such as the types in this library, the functions of the types, the parameters and return types of the functions, etc.
How does my code manufacture this DLL using techniques such as reflection?
I know my code can generate the "mytest.cs" text file, then execute the compiler to produce the DLL, then delete the "mytest.cs" file. Just want to know if there are "more advanced" or "cooler" ways to do it.
Thanks.

There are 4 main steps in the process to compile and execute dynamic .net scripts from your application, even really complex scenarios can be simplified in this way:
Generate the code
Compile the script
Load the assembly
Execute the code
Lets generate a simple Hello Generated C# World App right now!:
Create a method that will generate an assembly that has 1 class called HelloWorldApp, this class has 1 method called GenerateMessage it will have X input parameters that will be integers, it will return a CSV string of the arguments that were passed in to it.
This solution requires the following package to be installed:
PM> Install-Package 'Microsoft.CodeAnalysis.CSharp.Scripting'
And will require the following using statements:
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.Emit;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
Orchestration
The following method encapsulates the above steps:
private static void GenerateAndExecuteApp(int numberOfParameters)
{
string nameSpace = "Dynamic.Example";
string className = "HelloWorldApp";
string methodName = "GenerateMessage";
// 1. Generate the code
string script = BuildScript(nameSpace, className, methodName, numberOfParameters);
// 2. Compile the script
// 3. Load the Assembly
Assembly dynamicAssembly = CompileScript(script);
// 4. Execute the code
int[] arguments = Enumerable.Range(1, numberOfParameters).ToArray();
string message = ExecuteScript(dynamicAssembly, nameSpace, className, methodName, arguments);
Console.Out.WriteLine(message);
}
Generate the code
You say you already have item 1 sorted out, you can use StringBuilder, T4 templates or other mechanisms to generate the code files.
generating the code itself is its own question if you need help with that.
However, for our demo app, the following would work:
private static string BuildScript(string nameSpace, string className, string methodName, int numberOfParameters)
{
StringBuilder code = new StringBuilder();
code.AppendLine("using System;");
code.AppendLine("using System.Linq;");
code.AppendLine();
code.AppendLine($"namespace {nameSpace}");
code.AppendLine("{");
code.AppendLine($" public class {className}");
code.AppendLine(" {");
var parameterNames = Enumerable.Range(0, numberOfParameters).Select(x => $"p{x}").ToList();
code.Append($" public string {methodName}(");
code.Append(String.Join(",", parameterNames.Select(x => $"int {x}")));
code.AppendLine(")");
code.AppendLine(" {");
code.Append(" return $\"");
code.Append(String.Join(",", parameterNames.Select(x => $"{x}={{{x}}}")));
code.AppendLine("\";");
code.AppendLine(" }");
code.AppendLine(" }");
code.AppendLine("}");
return code.ToString();
}
For an input value of 3, the following code is generated:
using System;
using System.Linq;
namespace Dynamic.Example
{
public class HelloWorldApp
{
public string GenerateMessage(int p0,int p1,int p2)
{
return $"p0={p0},p1={p1},p2={p2}";
}
}
}
Compile the script (and Load it)
These are two discrete steps, however it is easiest to code them together in the same method, for this example we will ignore the generated dll and load the assembly directly into memory, that is generally the more likely use case for this type of scripting scenario anyway.
The hardest element of this is usually the referencing of the relevant dlls. There are a number of ways to achieve this, including loading all the dlls that are in the current executing context, I find a simple way to do this is to access the Assembly reference from the Type reference for the types we want to use inside the dynamic script:
List<string> dlls = new List<string> {
typeof(object).Assembly.Location,
typeof(Enumerable).Assembly.Location
};
Cut a long story short, this method compiles and loads the assembly into memory. It includes some crude compilation error handling, just to demonstrate how to do it:
private static Assembly CompileScript(string script)
{
SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(script);
// use "mytest.dll" if you want, random works well enough
string assemblyName = System.IO.Path.GetRandomFileName();
List<string> dlls = new List<string> {
typeof(object).Assembly.Location,
typeof(Enumerable).Assembly.Location
};
MetadataReference[] references = dlls.Distinct().Select(x => MetadataReference.CreateFromFile(x)).ToArray();
CSharpCompilation compilation = CSharpCompilation.Create(
assemblyName,
syntaxTrees: new[] { syntaxTree },
references: references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
// Now we actually compile the script, this includes some very crude error handling, just to show you can
using (var ms = new MemoryStream())
{
EmitResult result = compilation.Emit(ms);
if (!result.Success)
{
IEnumerable<Diagnostic> failures = result.Diagnostics.Where(diagnostic =>
diagnostic.IsWarningAsError ||
diagnostic.Severity == DiagnosticSeverity.Error);
List<string> errors = new List<string>();
foreach (Diagnostic diagnostic in failures)
{
//errors.AddDistinct(String.Format("{0} : {1}", diagnostic.Id, diagnostic.Location, diagnostic.GetMessage()));
errors.Add(diagnostic.ToString());
}
throw new ApplicationException("Compilation Errors: " + String.Join(Environment.NewLine, errors));
}
else
{
ms.Seek(0, SeekOrigin.Begin);
return Assembly.Load(ms.ToArray());
}
}
}
Execute the code
Finally, we can use reflection to instantiate an instance of the new app and then we can obtain a reference to the method and it. The name of the parameters is irrelevant, as long
we pass them through in the correct order:
for this demo the order is sort of irrelevant to, given they are all the same type ;)
private static string ExecuteScript(Assembly assembly, string nameSpace, string className, string methodName, int[] arguments)
{
var appType = assembly.GetType($"{nameSpace}.{className}");
object app = Activator.CreateInstance(appType);
MethodInfo method = appType.GetMethod(methodName);
object result = method.Invoke(app, arguments.Cast<object>().ToArray());
return result as string;
}
Output
The final output from all this for our method with 3 passed into it is:
p0=1,p1=2,p2=3
So that was super crude, you can bypass most of the indirect reflection aspects through the use of Interfaces. If your generated script inherits from types or interfaces that the calling code also has a strong reference to, then ExecuteScript in the above example might look like this:
private static string ExecuteScript(Assembly assembly, string nameSpace, string className)
{
var appType = assembly.GetType($"{nameSpace}.{className}");
object app = Activator.CreateInstance(appType);
if (app is KnownInterface known)
{
return known.GenerateMessage(1,2,3);
}
throw new NotSupportedException("Couldn't resolve known type");
}
The major benefit to using an interface or base class reference is that you can natively set properties or call other methods without having to reflect references to them all or to resort to using dynamic which would work, but becomes a bit harder to debug.
Of course the interface solution is hard to implement when we had a variable number of parameters, so that's not the best example, usually with dynamic scripts you would construct a known environment, say a known class and methods, but you might want to inject custom code into the body of the method.
It's a bit of fun in the end, but this simple example shows that C# can be used as a runtime scripting engine without too much trouble.

Assembly loading in Visual Studio extension

I'm writing a visual studio extension and I'm completely confused about how and where and when it's loading assemblies. What I have is this:
My Visual Studio extension project (let's call it MyExtension) references several assemblies including an assembly called Foo.dll.
MyExtension contains a class called FooManager that will be instantiated in response to a menu item being clicked.
When FooManager is instantiated it is passed the output path of a project in the current solution and it creates an AppDomain which should load that assembly, something like this:
public FooManager(string assemblyPath)
{
// The actual ApplicationBase of the current domain will be the one of VS and
// not of my plugin
// We need our new AppDomain to be able to find our assemblies
// without this even the CreateInstanceAndUnwrap will fail
var p = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var cachePath = Path.Combine(p, "Cache");
var pluginPath = Path.Combine(p, "Test");
if (Directory.Exists(cachePath))
{
Directory.Delete(cachePath, true);
}
if (Directory.Exists(pluginPath))
{
Directory.Delete(pluginPath, true);
}
Directory.CreateDirectory(cachePath);
Directory.CreateDirectory(pluginPath);
var newPath = Path.Combine(pluginPath, Path.GetFileName(assemblyPath));
File.Copy(assemblyPath, newPath, true);
var setup = new AppDomainSetup()
{
ApplicationBase = p,
ShadowCopyFiles = "true",
ShadowCopyDirectories = pluginPath,
CachePath = cachePath
};
domain = AppDomain.CreateDomain("MyTest_AppDomain", AppDomain.CurrentDomain.Evidence, setup);
// FooCollection is defined in MyExtension. but has several references
// to things defined in Foo.dll - it used MEF to load the assembly
// referenced by pluginPath
collection = domain.CreateInstanceAndUnwrap(
typeof(FooCollection).Assembly.FullName,
typeof(FooCollection).FullName,
false,
BindingFlags.Default,
null,
new object[] { pluginPath }, null, null) as FooCollection;
}
Now one property of FooManager looks like this (FooInfo is defined in
Foo.dll):
public IEnumerable<FooInfo> Spiders
{
get
{
return collection.Foos.Select(s => s.Metadata);
}
}
But when I try to access that I get a System.ArgumentException with the message Object type cannot be converted to target type. which I know happens if two copies of the same assembly are loaded from different locations, and I think, ultimately, that's what's happening here, but I can't figure out how to get it to load from the same place.
So after struggling with this a lot (and the above is only my latest attempt), I thought maybe I could serialize to byte[] and then deserialize again as a way to avoid the problem with types, so I tried something like this:
var msgBytes = collection.SerializeFooInfo();
var msg = FooInfo.DeserializeMessage(msgBytes);
Where my serialize and deserialize just use a BinaryFormatter (the classes are marked as Serializable). The serialization seems to work, but on deserialization, when I get to here:
public static List<FooInfo> DeserializeMessage(byte[] source)
{
using (var stream = new MemoryStream(source))
{
BinaryFormatter formatter = new BinaryFormatter();
var msg = formatter.Deserialize(stream);
return msg as List<FooInfo>;
}
}
msg comes back as null. If I try to run it in the debugger using the immediate window, I see that Deserialize threw a FileNotFoundException with the message:
Cannot load assembly 'C:\Users\matt.burland\AppData\Local\Microsoft\VisualStudio\14.0Exp\ProjectAssemblies\qesxy6ms01\Foo.dll'
But I don't understand where that path came from. It's not where my extension has been installed, which is C:\Users\matt.burland\AppData\Local\Microsoft\VisualStudio\14.0Exp\Extensions\MyCompany\FooTools\1.0 and was set as the ApplicationBase for my AppDomain and contains the file foo.dll. So why is it trying to load from the other mystery location? The other location seems to be created dynamically and contains only the foo.dll assembly.
I've do something very similar to this in a windows service (and using a lot of the same classes) and it works just fine, so this seems to be something particular to the way Visual Studio extensions. Can anybody shine a light here?
So I thought this might help: http://geekswithblogs.net/onlyutkarsh/archive/2013/06/02/loading-custom-assemblies-in-visual-studio-extensions-again.aspx
But if I try to attach an AssemblyResolve handler in package class as suggested it doesn't get called for anything interesting (which isn't surprising, it's not the domain I'm try to load from), but if I try to attach to the new domain I create then if I try something like this:
domain.AssemblyResolve += OnAssemblyResolve;
Then it fails because my FooManager isn't marked as serializable. So I created a proxy just for binding the AssemblyResolve, but the AssemblyResolve never fires. So I tried not setting the ApplicationBase when creating my domain, thinking that would force it to have to try to resolve, but then I can't create my proxy class in the created domain because it doesn't know where to load the assembly from!.

Deserialization fail when using themida protection

As the title says the deserialization fail after protecting my application with themida with the following exception :
Unable to generate a temporary class (result=1).
error CS0009: Metadata file 'c:\Path\to\protected.exe' could not be opened -- 'An attempt was made to load a program with an incorrect format. '
Here's the code I'm using for the deserialization (It works when the exe is not protected):
MyClass myClass;
try
{
using (var stream = new MemoryStream(Data))
{
var serializer = new XmlSerializer(typeof(ComSec.MyClass));
myClass = serializer.Deserialize(stream) as MyClass;
}
}
catch (Exception e)
{
return null;
}
Weird thing is that the code + themida protection works fine on my machine but it fails on the VM and on a co-worker's machine
I'm using (same config as my co-worker) :
VS2012 Professional
Windows 7 x64 Ultimate
Themida 2.1.2.0 x86 (With .Net support)
The VM is a fresh install of Windows 7 x86.

I ended up using the DataContract attribute and using a DataContractSerializer to Serialize and deserialize the object (It works now everywhere and with/without the protection ).
My research:
[DataContract(Name = "TestClass")]
public class TestClass
{
[DataMember(Name = "Name")]
public string Name { get; set; }
[DataMember(Name = "Age")]
public int Age { get; set; }
}
Serialization/Deserialization :
var serializer = new DataContractSerializer(typeof(TestClass));
using (var stream = new MemoryStream())
{
serializer.WriteObject(stream, this);
File.WriteAllBytes("TestClass.xml", stream.ToArray());
}
TestClass o = null;
using (var stream = new MemoryStream(File.ReadAllBytes("TestClass.xml")))
{
o = serializer.ReadObject(stream) as TestClass;
}

I experienced this issue as well and the proposed answer worked fine. The original problem was due to file access permissions. Just adding this information so other people understand why DataContract works (from https://stackoverflow.com/a/10340155/1111380):
DataContractSerializer, NetDataContractSerializer and DataContractJsonSerializer do NOT require disk space and do NOT emit assemblies to disk. Instead, they generate IL on the fly (in memory) and use it during subsequent serialization episodes to do serialization and deserialization all within the AppDomain they're operating in. However, XmlSerializer does require disk space and explains the error (path to the file could not be opened/access).

How do I use XmlSerializer to handle different namespace versions?

I am using the .NET XmlSerializer class to deserialize GPX files.
There are two versions of the GPX standard:
<gpx xmlns="http://www.topografix.com/GPX/1/0"> ... </gpx>
<gpx xmlns="http://www.topografix.com/GPX/1/1"> ... </gpx>
Also, some GPX files do not specify a default namespace:
<gpx> ... </gpx>
My code needs to handle all three cases, but I can't work out how to get XmlSerializer to do it.
I am sure there must be a simple solution because this a common scenario, for example KML has the same issue.

I have done something similar to this a few times before, and this might be of use to you if you only have to deal with a small number of namespaces and you know them all beforehand. Create a simple inheritance hierarchy of classes, and add attributes to the different classes for the different namespaces. See the following code sample. If you run this program it gives the output:
Deserialized, type=XmlSerializerExample.GpxV1, data=1
Deserialized, type=XmlSerializerExample.GpxV2, data=2
Deserialized, type=XmlSerializerExample.Gpx, data=3
Here is the code:
using System;
using System.IO;
using System.Xml;
using System.Xml.Serialization;
[XmlRoot("gpx")]
public class Gpx {
[XmlElement("data")] public int Data;
}
[XmlRoot("gpx", Namespace = "http://www.topografix.com/GPX/1/0")]
public class GpxV1 : Gpx {}
[XmlRoot("gpx", Namespace = "http://www.topografix.com/GPX/1/1")]
public class GpxV2 : Gpx {}
internal class Program {
private static void Main() {
var xmlExamples = new[] {
"<gpx xmlns='http://www.topografix.com/GPX/1/0'><data>1</data></gpx>",
"<gpx xmlns='http://www.topografix.com/GPX/1/1'><data>2</data></gpx>",
"<gpx><data>3</data></gpx>",
};
var serializers = new[] {
new XmlSerializer(typeof (Gpx)),
new XmlSerializer(typeof (GpxV1)),
new XmlSerializer(typeof (GpxV2)),
};
foreach (var xml in xmlExamples) {
var textReader = new StringReader(xml);
var xmlReader = XmlReader.Create(textReader);
foreach (var serializer in serializers) {
if (serializer.CanDeserialize(xmlReader)) {
var gpx = (Gpx)serializer.Deserialize(xmlReader);
Console.WriteLine("Deserialized, type={0}, data={1}", gpx.GetType(), gpx.Data);
}
}
}
}
}

Here's the solution I came up with before the other suggestions came through:
var settings = new XmlReaderSettings();
settings.IgnoreComments = true;
settings.IgnoreProcessingInstructions = true;
settings.IgnoreWhitespace = true;
using (var reader = XmlReader.Create(filePath, settings))
{
if (reader.IsStartElement("gpx"))
{
string defaultNamespace = reader["xmlns"];
XmlSerializer serializer = new XmlSerializer(typeof(Gpx), defaultNamespace);
gpx = (Gpx)serializer.Deserialize(reader);
}
}
This example accepts any namespace, but you could easily make it filter for a specific list of known namespaces.

Oddly enough you can't solve this nicely. Have a look at the deserialize section in this troubleshooting article. Especially where it states:
Only a few error conditions lead to exceptions during the
deserialization process. The most common ones are:
•The name of the
root element or its namespace did not match the expected name.
...
The workaround I use for this is to set the first namespace, try/catch the deserialize operation and if it fails because of the namespace I try it with the next one. Only if all namespace options fail do I throw the error.
From a really strict point of view you can argue that this behavior is correct since the type you deserialize to should represent a specific schema/namespace and then it doesn't make sense that it should also be able to read data from another schema/namespace. In practice this is utterly annoying though. File extenstion rarely change when versions change so the only way to tell if a .gpx file is v0 or v1 is to read the xml contents but the xmldeserializer won't unless you tell upfront which version it will be.

How can I serialize internal classes using XmlSerializer?

I'm building a library to interface with a third party. Communication is through XML and HTTP Posts. That's working.
But, whatever code uses the library does not need to be aware of the internal classes. My internal objects are serialized to XML using this method:
internal static string SerializeXML(Object obj)
{
XmlSerializer serializer = new XmlSerializer(obj.GetType(), "some.domain");
//settings
XmlWriterSettings settings = new XmlWriterSettings();
settings.Indent = true;
settings.OmitXmlDeclaration = true;
using (StringWriter stream = new StringWriter())
{
using (XmlWriter writer = XmlWriter.Create(stream, settings))
{
serializer.Serialize(writer, obj);
}
return stream.ToString();
}
}
However, when I change my classes' access modifier to internal, I get an exception at runtime:
[System.InvalidOperationException] = {"MyNamespace.MyClass is inaccessible due to its protection level. Only public types can be processed."}
That exception happens in the first line of the code above.
I would like my library's classes not to be public because I do not want to expose them. Can I do that? How can I make internal types serializable, using my generic serializer? What am I doing wrong?

From Sowmy Srinivasan's Blog - Serializing internal types using XmlSerializer:
Being able to serialize internal types is one of the common requests
seen by the XmlSerializer team. It is a reasonable request from people
shipping libraries. They do not want to make the XmlSerializer types
public just for the sake of the serializer. I recently moved from the
team that wrote the XmlSerializer to a team that consumes
XmlSerializer. When I came across a similar request I said, "No way.
Use DataContractSerializer".
The reason is simple. XmlSerializer works by generating code. The
generated code lives in a dynamically generated assembly and needs to
access the types being serialized. Since XmlSerializer was developed
in a time before the advent of lightweight code generation, the
generated code cannot access anything other than public types in
another assembly. Hence the types being serialized has to be public.
I hear astute readers whisper "It does not have to be public if
'InternalsVisibleTo' attribute is used".
I say, "Right, but the name of the generated assembly is not known
upfront. To which assembly do you make the internals visible to?"
Astute readers : "the assembly name is known if one uses 'sgen.exe'"
Me: "For sgen to generate serializer for your types, they have to be
public"
Astute readers : "We could do a two pass compilation. One pass for
sgen with types as public and another pass for shipping with types as
internals."
They may be right! If I ask the astute readers to write me a sample
they would probably write something like this. (Disclaimer: This is
not the official solution. YMMV)
using System;
using System.IO;
using System.Xml.Serialization;
using System.Runtime.CompilerServices;
using System.Reflection;
[assembly: InternalsVisibleTo("Program.XmlSerializers")]
namespace InternalTypesInXmlSerializer
{
class Program
{
static void Main(string[] args)
{
Address address = new Address();
address.Street = "One Microsoft Way";
address.City = "Redmond";
address.Zip = 98053;
Order order = new Order();
order.BillTo = address;
order.ShipTo = address;
XmlSerializer xmlSerializer = GetSerializer(typeof(Order));
xmlSerializer.Serialize(Console.Out, order);
}
static XmlSerializer GetSerializer(Type type)
{
#if Pass1
return new XmlSerializer(type);
#else
Assembly serializersDll = Assembly.Load("Program.XmlSerializers");
Type xmlSerializerFactoryType = serializersDll.GetType("Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializerContract");
MethodInfo getSerializerMethod = xmlSerializerFactoryType.GetMethod("GetSerializer", BindingFlags.Public | BindingFlags.Instance);
return (XmlSerializer)getSerializerMethod.Invoke(Activator.CreateInstance(xmlSerializerFactoryType), new object[] { type });
#endif
}
}
#if Pass1
public class Address
#else
internal class Address
#endif
{
public string Street;
public string City;
public int Zip;
}
#if Pass1
public class Order
#else
internal class Order
#endif
{
public Address ShipTo;
public Address BillTo;
}
}
Some astute 'hacking' readers may go as far as giving me the build.cmd
to compile it.
csc /d:Pass1 program.cs
sgen program.exe
csc program.cs

As an alternative you can use dynamically created public classes (which won't be exposed to the 3rd party):
static void Main()
{
var emailType = CreateEmailType();
dynamic email = Activator.CreateInstance(emailType);
email.From = "x#xpto.com";
email.To = "y#acme.com";
email.Subject = "Dynamic Type";
email.Boby = "XmlSerializer can use this!";
}
static Type CreateEmailType()
{
var assemblyName = new AssemblyName("DynamicAssembly");
var assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run);
var moduleBuilder = assemblyBuilder.DefineDynamicModule(assemblyName.Name);
var typeBuilder = moduleBuilder.DefineType(
"Email",
(
TypeAttributes.Public |
TypeAttributes.Sealed |
TypeAttributes.SequentialLayout |
TypeAttributes.Serializable
),
typeof(ValueType)
);
typeBuilder.DefineField("From", typeof(string), FieldAttributes.Public);
typeBuilder.DefineField("To", typeof(string), FieldAttributes.Public);
typeBuilder.DefineField("Subject", typeof(string), FieldAttributes.Public);
typeBuilder.DefineField("Body", typeof(string), FieldAttributes.Public);
return typeBuilder.CreateType();
}

The no headache solution is to use the NetBike.Xml NuGet package (.NET Standard 2.0) which is compatible with System.Xml.XmlSerializer XML attributes and support internal types out of the box.
Usage is straightforward, here's an excerpt from the README:
var serializer = new XmlSerializer();
var xml = "<Foo><Id>1</Id><Name>test</Name></Foo>";
var foo = serializer.Deserialize<Foo>(new StringReader(xml));

This may help you: MRB_ObjectSaver
This project helps you save/load/clone any object in c# to/from a file/string. In compare to "c# serialization" this method keeps reference to objects and link between objects will not break. (see: SerializeObjectTest.cs for an example) Furthermore, the type have not be marked as [Serializable]

You can also use something called xgenplus (http://xgenplus.codeplex.com/) this generates the code that would normally get executed at runtime. Then you add that to your solution, and compile it as part of your solution. At that point, it doesn't matter if your object is internal - you can add the pre-generated code in the same namespace. The performance for this is blazing fast, as its all pre-generated.