First of all I know I shouldn't even try to do it, but i like to try things that don't make sense. I was intrigued by this post and modified to my needs and I wanted to improve it by implementing allocation of arrays inside unmanaged memory.
public static class Unmanaged<T> where T : class
{
private delegate T CreateHandler(IntPtr ptr);
private delegate IntPtr FindHandler(T obj);
private static readonly CreateHandler Create;
private static readonly FindHandler Find;
private static readonly IntPtr _typePointer;
private static readonly int _typeSize;
static Unmanaged()
{
Type type = typeof(T);
_typePointer = type.TypeHandle.Value;
_typeSize = Marshal.ReadInt32(_typePointer, sizeof(int));
DynamicMethod method = new DynamicMethod(nameof(Create), typeof(T), new[] { typeof(IntPtr) }, typeof(Unmanaged<T>), true);
var generator = method.GetILGenerator();
generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Ret);
Create = (CreateHandler)method.CreateDelegate(typeof(CreateHandler));
method = new DynamicMethod(nameof(Find), typeof(IntPtr), new[] { typeof(T) }, typeof(Unmanaged<T>), true);
generator = method.GetILGenerator();
generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Ret);
Find = (FindHandler)method.CreateDelegate(typeof(FindHandler));
}
public static T New()
{
IntPtr handle = Marshal.AllocHGlobal(_typeSize);
IntPtr pointer = handle + IntPtr.Size;
Marshal.WriteIntPtr(pointer, _typePointer);
return Create(pointer);
}
public static void Destroy(T obj)
{
IntPtr pointer = Find(obj);
IntPtr handle = pointer - IntPtr.Size;
Marshal.FreeHGlobal(handle);
}
}
public static class UnmanagedArray<T>
{
private static readonly Delegate Create;
private static readonly Delegate Find;
private static readonly IntPtr _typePointer;
private static readonly int _typeSize;
static UnmanagedArray()
{
Type type = typeof(T[]);
Type createType = Expression.GetFuncType(typeof(IntPtr), typeof(T[]));
Type findType = Expression.GetFuncType(typeof(T[]), typeof(IntPtr));
_typePointer = type.TypeHandle.Value;
_typeSize = Marshal.ReadInt32(_typePointer, sizeof(int));
DynamicMethod method = new DynamicMethod(nameof(Create), typeof(T[]), new[] { typeof(IntPtr) }, typeof(UnmanagedArray<T>), true);
ILGenerator generator = method.GetILGenerator();
generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Ret);
Create = method.CreateDelegate(createType);
method = new DynamicMethod(nameof(Find), typeof(IntPtr), new[] { typeof(T[]) }, typeof(UnmanagedArray<T>), true);
generator = method.GetILGenerator();
generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Ret);
Find = method.CreateDelegate(findType);
}
public static T[] New(int count)
{
Type elementType = typeof(T);
int elementSize = GetElementSize(elementType);
int size = _typeSize + elementSize * count;
IntPtr handle = Marshal.AllocHGlobal(size);
IntPtr pointer = handle + IntPtr.Size;
Marshal.WriteIntPtr(pointer, _typePointer);
InitializeArray(handle, size, count);
return (T[])Create.DynamicInvoke(pointer);
}
public static void Destroy(T[] obj)
{
IntPtr pointer = (IntPtr)Find.DynamicInvoke(obj);
IntPtr handle = pointer - IntPtr.Size;
Marshal.FreeHGlobal(handle);
}
private static int GetElementSize(Type type)
{
if (type.IsValueType) return Marshal.SizeOf(type);
else return IntPtr.Size;
}
private static unsafe void InitializeArray(IntPtr handle, int size, int count)
{
int startPosition = IntPtr.Size * 2;
int endPosition = IntPtr.Size * 2 + sizeof(int);
byte byteSize = 8;
byte* bytes = (byte*)handle;
for (int index = startPosition, positionIndex = 0; index < endPosition; index++, positionIndex++)
{
bytes[index] = (byte)(count >> positionIndex * byteSize);
}
for (int index = _typeSize; index < size; index++)
{
bytes[index] = 0;
}
}
}
For value types (short, int, long etc.) it works (I haven't tried struct's yet), but it crashes after allocating array for objects as below...
static void Main(string[] args)
{
Test test = Unmanaged<Test>.New();
Test[] array = UnmanagedArray<Test>.New(2);
test.Value = 5;
array[0] = test;
Console.WriteLine(array[0].Value); //AccessViolationException
Unmanaged<Test>.Destroy(test);
UnmanagedArray<Test>.Destroy(array);
}
Don't know why it crashes with AccessViolationException (which You can't catch by normal ways) and the best part is that it's something that not always occurs. Checked stack when debugging and the array actually saves reference to object in unmanaged memory (even checked addresses and they match), but later on it crashes (very often)...
Any suggestions?
Update
Turns out You can't assign or read nested reference* when allocating unmanaged memory for reference Type. Probably it's related to how GC manages references (don't know for sure, it's just a guess). That's why in the link above the author encapsulated reference inside struct. That way You're working with value type and if You change array to store those structs (containing Your unmanaged references) it will work.
*By nested references I mean fields in class which You try to allocate space in unmanaged memory and array elements (array of objects itself is a reference on stack to actual array on heap - if it's not a field in a class - which contains "nested" references as elements to objects - also on heap)
GC won't be able to update pointers to other objects inside your unmanaged memory and doesn't count them as live references, so any pointer to managed memory inside your object will be either invalid or dead (twice invalid). As Marc said, the only thing you can place in unmaged memory is structs adhering to unmanaged constraint.
Related
What I want to do is change how a C# method executes when it is called, so that I can write something like this:
[Distributed]
public DTask<bool> Solve(int n, DEvent<bool> callback)
{
for (int m = 2; m < n - 1; m += 1)
if (m % n == 0)
return false;
return true;
}
At run-time, I need to be able to analyse methods that have the Distributed attribute (which I already can do) and then insert code before the body of the function executes and after the function returns. More importantly, I need to be able to do it without modifying code where Solve is called or at the start of the function (at compile time; doing so at run-time is the objective).
At the moment I have attempted this bit of code (assume t is the type that Solve is stored in, and m is a MethodInfo of Solve):
private void WrapMethod(Type t, MethodInfo m)
{
// Generate ILasm for delegate.
byte[] il = typeof(Dpm).GetMethod("ReplacedSolve").GetMethodBody().GetILAsByteArray();
// Pin the bytes in the garbage collection.
GCHandle h = GCHandle.Alloc((object)il, GCHandleType.Pinned);
IntPtr addr = h.AddrOfPinnedObject();
int size = il.Length;
// Swap the method.
MethodRental.SwapMethodBody(t, m.MetadataToken, addr, size, MethodRental.JitImmediate);
}
public DTask<bool> ReplacedSolve(int n, DEvent<bool> callback)
{
Console.WriteLine("This was executed instead!");
return true;
}
However, MethodRental.SwapMethodBody only works on dynamic modules; not those that have already been compiled and stored in the assembly.
So I'm looking for a way to effectively do SwapMethodBody on a method that is already stored in a loaded and executing assembly.
Note, it is not an issue if I have to completely copy the method into a dynamic module, but in this case I need to find a way to copy across the IL as well as update all of the calls to Solve() such that they would point to the new copy.
Disclosure: Harmony is a library that was written and is maintained by me, the author of this post.
Harmony 2 is an open source library (MIT license) designed to replace, decorate or modify existing C# methods of any kind during runtime. Its main focus is games and plugins written in Mono or .NET. It takes care of multiple changes to the same method - they accumulate instead of overwrite each other.
It creates dynamic replacement methods for every original method and emits code to them that calls custom methods at the start and end. It also allows you to write filters to process the original IL code and custom exception handlers which allows for more detailed manipulation of the original method.
To complete the process, it writes a simple assembler jump into the trampoline of the original method that points to the assembler generated from compiling the dynamic method. This works for 32/64-bit on Windows, macOS and any Linux that Mono supports.
Documentation can be found here.
Example
(Source)
Original Code
public class SomeGameClass
{
private bool isRunning;
private int counter;
private int DoSomething()
{
if (isRunning)
{
counter++;
return counter * 10;
}
}
}
Patching with Harmony annotations
using SomeGame;
using HarmonyLib;
public class MyPatcher
{
// make sure DoPatching() is called at start either by
// the mod loader or by your injector
public static void DoPatching()
{
var harmony = new Harmony("com.example.patch");
harmony.PatchAll();
}
}
[HarmonyPatch(typeof(SomeGameClass))]
[HarmonyPatch("DoSomething")]
class Patch01
{
static FieldRef<SomeGameClass,bool> isRunningRef =
AccessTools.FieldRefAccess<SomeGameClass, bool>("isRunning");
static bool Prefix(SomeGameClass __instance, ref int ___counter)
{
isRunningRef(__instance) = true;
if (___counter > 100)
return false;
___counter = 0;
return true;
}
static void Postfix(ref int __result)
{
__result *= 2;
}
}
Alternatively, manual patching with reflection
using SomeGame;
using System.Reflection;
using HarmonyLib;
public class MyPatcher
{
// make sure DoPatching() is called at start either by
// the mod loader or by your injector
public static void DoPatching()
{
var harmony = new Harmony("com.example.patch");
var mOriginal = typeof(SomeGameClass).GetMethod("DoSomething", BindingFlags.Instance | BindingFlags.NonPublic);
var mPrefix = typeof(MyPatcher).GetMethod("MyPrefix", BindingFlags.Static | BindingFlags.Public);
var mPostfix = typeof(MyPatcher).GetMethod("MyPostfix", BindingFlags.Static | BindingFlags.Public);
// add null checks here
harmony.Patch(mOriginal, new HarmonyMethod(mPrefix), new HarmonyMethod(mPostfix));
}
public static void MyPrefix()
{
// ...
}
public static void MyPostfix()
{
// ...
}
}
For .NET 4 and above
using System;
using System.Reflection;
using System.Runtime.CompilerServices;
namespace InjectionTest
{
class Program
{
static void Main(string[] args)
{
Target targetInstance = new Target();
targetInstance.test();
Injection.install(1);
Injection.install(2);
Injection.install(3);
Injection.install(4);
targetInstance.test();
Console.Read();
}
}
public class Target
{
public void test()
{
targetMethod1();
Console.WriteLine(targetMethod2());
targetMethod3("Test");
targetMethod4();
}
private void targetMethod1()
{
Console.WriteLine("Target.targetMethod1()");
}
private string targetMethod2()
{
Console.WriteLine("Target.targetMethod2()");
return "Not injected 2";
}
public void targetMethod3(string text)
{
Console.WriteLine("Target.targetMethod3("+text+")");
}
private void targetMethod4()
{
Console.WriteLine("Target.targetMethod4()");
}
}
public class Injection
{
public static void install(int funcNum)
{
MethodInfo methodToReplace = typeof(Target).GetMethod("targetMethod"+ funcNum, BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
MethodInfo methodToInject = typeof(Injection).GetMethod("injectionMethod"+ funcNum, BindingFlags.Instance | BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
RuntimeHelpers.PrepareMethod(methodToReplace.MethodHandle);
RuntimeHelpers.PrepareMethod(methodToInject.MethodHandle);
unsafe
{
if (IntPtr.Size == 4)
{
int* inj = (int*)methodToInject.MethodHandle.Value.ToPointer() + 2;
int* tar = (int*)methodToReplace.MethodHandle.Value.ToPointer() + 2;
#if DEBUG
Console.WriteLine("\nVersion x86 Debug\n");
byte* injInst = (byte*)*inj;
byte* tarInst = (byte*)*tar;
int* injSrc = (int*)(injInst + 1);
int* tarSrc = (int*)(tarInst + 1);
*tarSrc = (((int)injInst + 5) + *injSrc) - ((int)tarInst + 5);
#else
Console.WriteLine("\nVersion x86 Release\n");
*tar = *inj;
#endif
}
else
{
long* inj = (long*)methodToInject.MethodHandle.Value.ToPointer()+1;
long* tar = (long*)methodToReplace.MethodHandle.Value.ToPointer()+1;
#if DEBUG
Console.WriteLine("\nVersion x64 Debug\n");
byte* injInst = (byte*)*inj;
byte* tarInst = (byte*)*tar;
int* injSrc = (int*)(injInst + 1);
int* tarSrc = (int*)(tarInst + 1);
*tarSrc = (((int)injInst + 5) + *injSrc) - ((int)tarInst + 5);
#else
Console.WriteLine("\nVersion x64 Release\n");
*tar = *inj;
#endif
}
}
}
private void injectionMethod1()
{
Console.WriteLine("Injection.injectionMethod1");
}
private string injectionMethod2()
{
Console.WriteLine("Injection.injectionMethod2");
return "Injected 2";
}
private void injectionMethod3(string text)
{
Console.WriteLine("Injection.injectionMethod3 " + text);
}
private void injectionMethod4()
{
System.Diagnostics.Process.Start("calc");
}
}
}
You CAN modify a method's content at runtime. But you're not supposed to, and it's strongly recommended to keep that for test purposes.
Just have a look at:
http://www.codeproject.com/Articles/463508/NET-CLR-Injection-Modify-IL-Code-during-Run-time
Basically, you can:
Get IL method content via MethodInfo.GetMethodBody().GetILAsByteArray()
Mess with these bytes.
If you just wish to prepend or append some code, then just preprend/append opcodes you want (be careful about leaving stack clean, though)
Here are some tips to "uncompile" existing IL:
Bytes returned are a sequence of IL instructions, followed by their arguments (if they have some - for instance, '.call' has one argument: the called method token, and '.pop' has none)
Correspondence between IL codes and bytes you find in the returned array may be found using OpCodes.YourOpCode.Value (which is the real opcode byte value as saved in your assembly)
Arguments appended after IL codes may have different sizes (from one to several bytes), depending on opcode called
You may find tokens that theses arguments are referring to via appropriate methods. For instance, if your IL contains ".call 354354" (coded as 28 00 05 68 32 in hexa, 28h=40 being '.call' opcode and 56832h=354354), corresponding called method can be found using MethodBase.GetMethodFromHandle(354354)
Once modified, you IL byte array can be reinjected via InjectionHelper.UpdateILCodes(MethodInfo method, byte[] ilCodes) - see link mentioned above
This is the "unsafe" part... It works well, but this consists in hacking internal CLR mechanisms...
you can replace it if the method is non virtual, non generic, not in generic type, not inlined and on x86 plateform:
MethodInfo methodToReplace = ...
RuntimeHelpers.PrepareMetod(methodToReplace.MethodHandle);
var getDynamicHandle = Delegate.CreateDelegate(Metadata<Func<DynamicMethod, RuntimeMethodHandle>>.Type, Metadata<DynamicMethod>.Type.GetMethod("GetMethodDescriptor", BindingFlags.Instance | BindingFlags.NonPublic)) as Func<DynamicMethod, RuntimeMethodHandle>;
var newMethod = new DynamicMethod(...);
var body = newMethod.GetILGenerator();
body.Emit(...) // do what you want.
body.Emit(OpCodes.jmp, methodToReplace);
body.Emit(OpCodes.ret);
var handle = getDynamicHandle(newMethod);
RuntimeHelpers.PrepareMethod(handle);
*((int*)new IntPtr(((int*)methodToReplace.MethodHandle.Value.ToPointer() + 2)).ToPointer()) = handle.GetFunctionPointer().ToInt32();
//all call on methodToReplace redirect to newMethod and methodToReplace is called in newMethod and you can continue to debug it, enjoy.
Based on the answer to this question and another, ive came up with this tidied up version:
// Note: This method replaces methodToReplace with methodToInject
// Note: methodToInject will still remain pointing to the same location
public static unsafe MethodReplacementState Replace(this MethodInfo methodToReplace, MethodInfo methodToInject)
{
//#if DEBUG
RuntimeHelpers.PrepareMethod(methodToReplace.MethodHandle);
RuntimeHelpers.PrepareMethod(methodToInject.MethodHandle);
//#endif
MethodReplacementState state;
IntPtr tar = methodToReplace.MethodHandle.Value;
if (!methodToReplace.IsVirtual)
tar += 8;
else
{
var index = (int)(((*(long*)tar) >> 32) & 0xFF);
var classStart = *(IntPtr*)(methodToReplace.DeclaringType.TypeHandle.Value + (IntPtr.Size == 4 ? 40 : 64));
tar = classStart + IntPtr.Size * index;
}
var inj = methodToInject.MethodHandle.Value + 8;
#if DEBUG
tar = *(IntPtr*)tar + 1;
inj = *(IntPtr*)inj + 1;
state.Location = tar;
state.OriginalValue = new IntPtr(*(int*)tar);
*(int*)tar = *(int*)inj + (int)(long)inj - (int)(long)tar;
return state;
#else
state.Location = tar;
state.OriginalValue = *(IntPtr*)tar;
* (IntPtr*)tar = *(IntPtr*)inj;
return state;
#endif
}
}
public struct MethodReplacementState : IDisposable
{
internal IntPtr Location;
internal IntPtr OriginalValue;
public void Dispose()
{
this.Restore();
}
public unsafe void Restore()
{
#if DEBUG
*(int*)Location = (int)OriginalValue;
#else
*(IntPtr*)Location = OriginalValue;
#endif
}
}
There exists a couple of frameworks that allows you to dynamically change any method at runtime (they use the ICLRProfiling interface mentioned by user152949):
Prig: Free and Open Source!
Microsoft Fakes: Commercial, included in Visual Studio Premium and Ultimate but not Community and Professional
Telerik JustMock: Commercial, a "lite" version is available
Typemock Isolator: Commercial
There are also a few frameworks that mocks around with the internals of .NET, these are likely more fragile, and probably can't change inlined code, but on the other hand they are fully self-contained and does not require you to use a custom launcher.
Harmony: MIT licensed. Seems to actually have been used sucessfully in a few game mods, supports both .NET and Mono.
Deviare In Process Instrumentation Engine: GPLv3 and Commercial. .NET support currently marked as experimental, but on the other hand has the benefit of being commercially backed.
Logman's solution, but with an interface for swapping method bodies. Also, a simpler example.
using System;
using System.Linq;
using System.Reflection;
using System.Runtime.CompilerServices;
namespace DynamicMojo
{
class Program
{
static void Main(string[] args)
{
Animal kitty = new HouseCat();
Animal lion = new Lion();
var meow = typeof(HouseCat).GetMethod("Meow", BindingFlags.Instance | BindingFlags.NonPublic);
var roar = typeof(Lion).GetMethod("Roar", BindingFlags.Instance | BindingFlags.NonPublic);
Console.WriteLine("<==(Normal Run)==>");
kitty.MakeNoise(); //HouseCat: Meow.
lion.MakeNoise(); //Lion: Roar!
Console.WriteLine("<==(Dynamic Mojo!)==>");
DynamicMojo.SwapMethodBodies(meow, roar);
kitty.MakeNoise(); //HouseCat: Roar!
lion.MakeNoise(); //Lion: Meow.
Console.WriteLine("<==(Normality Restored)==>");
DynamicMojo.SwapMethodBodies(meow, roar);
kitty.MakeNoise(); //HouseCat: Meow.
lion.MakeNoise(); //Lion: Roar!
Console.Read();
}
}
public abstract class Animal
{
public void MakeNoise() => Console.WriteLine($"{this.GetType().Name}: {GetSound()}");
protected abstract string GetSound();
}
public sealed class HouseCat : Animal
{
protected override string GetSound() => Meow();
private string Meow() => "Meow.";
}
public sealed class Lion : Animal
{
protected override string GetSound() => Roar();
private string Roar() => "Roar!";
}
public static class DynamicMojo
{
/// <summary>
/// Swaps the function pointers for a and b, effectively swapping the method bodies.
/// </summary>
/// <exception cref="ArgumentException">
/// a and b must have same signature
/// </exception>
/// <param name="a">Method to swap</param>
/// <param name="b">Method to swap</param>
public static void SwapMethodBodies(MethodInfo a, MethodInfo b)
{
if (!HasSameSignature(a, b))
{
throw new ArgumentException("a and b must have have same signature");
}
RuntimeHelpers.PrepareMethod(a.MethodHandle);
RuntimeHelpers.PrepareMethod(b.MethodHandle);
unsafe
{
if (IntPtr.Size == 4)
{
int* inj = (int*)b.MethodHandle.Value.ToPointer() + 2;
int* tar = (int*)a.MethodHandle.Value.ToPointer() + 2;
byte* injInst = (byte*)*inj;
byte* tarInst = (byte*)*tar;
int* injSrc = (int*)(injInst + 1);
int* tarSrc = (int*)(tarInst + 1);
int tmp = *tarSrc;
*tarSrc = (((int)injInst + 5) + *injSrc) - ((int)tarInst + 5);
*injSrc = (((int)tarInst + 5) + tmp) - ((int)injInst + 5);
}
else
{
throw new NotImplementedException($"{nameof(SwapMethodBodies)} doesn't yet handle IntPtr size of {IntPtr.Size}");
}
}
}
private static bool HasSameSignature(MethodInfo a, MethodInfo b)
{
bool sameParams = !a.GetParameters().Any(x => !b.GetParameters().Any(y => x == y));
bool sameReturnType = a.ReturnType == b.ReturnType;
return sameParams && sameReturnType;
}
}
}
Based on TakeMeAsAGuest's answer, here's a similar extension which does not require to use unsafe blocks.
Here's the Extensions class:
using System;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
namespace MethodRedirect
{
static class Extensions
{
public static void RedirectTo(this MethodInfo origin, MethodInfo target)
{
IntPtr ori = GetMethodAddress(origin);
IntPtr tar = GetMethodAddress(target);
Marshal.Copy(new IntPtr[] { Marshal.ReadIntPtr(tar) }, 0, ori, 1);
}
private static IntPtr GetMethodAddress(MethodInfo mi)
{
const ushort SLOT_NUMBER_MASK = 0xffff; // 2 bytes mask
const int MT_OFFSET_32BIT = 0x28; // 40 bytes offset
const int MT_OFFSET_64BIT = 0x40; // 64 bytes offset
IntPtr address;
// JIT compilation of the method
RuntimeHelpers.PrepareMethod(mi.MethodHandle);
IntPtr md = mi.MethodHandle.Value; // MethodDescriptor address
IntPtr mt = mi.DeclaringType.TypeHandle.Value; // MethodTable address
if (mi.IsVirtual)
{
// The fixed-size portion of the MethodTable structure depends on the process type
int offset = IntPtr.Size == 4 ? MT_OFFSET_32BIT : MT_OFFSET_64BIT;
// First method slot = MethodTable address + fixed-size offset
// This is the address of the first method of any type (i.e. ToString)
IntPtr ms = Marshal.ReadIntPtr(mt + offset);
// Get the slot number of the virtual method entry from the MethodDesc data structure
long shift = Marshal.ReadInt64(md) >> 32;
int slot = (int)(shift & SLOT_NUMBER_MASK);
// Get the virtual method address relative to the first method slot
address = ms + (slot * IntPtr.Size);
}
else
{
// Bypass default MethodDescriptor padding (8 bytes)
// Reach the CodeOrIL field which contains the address of the JIT-compiled code
address = md + 8;
}
return address;
}
}
}
And here's a simple usage example:
using System;
using System.Reflection;
namespace MethodRedirect
{
class Scenario
{
static void Main(string[] args)
{
Assembly assembly = Assembly.GetAssembly(typeof(Scenario));
Type Scenario_Type = assembly.GetType("MethodRedirect.Scenario");
MethodInfo Scenario_InternalInstanceMethod = Scenario_Type.GetMethod("InternalInstanceMethod", BindingFlags.Instance | BindingFlags.NonPublic);
MethodInfo Scenario_PrivateInstanceMethod = Scenario_Type.GetMethod("PrivateInstanceMethod", BindingFlags.Instance | BindingFlags.NonPublic);
Scenario_InternalInstanceMethod.RedirectTo(Scenario_PrivateInstanceMethod);
// Using dynamic type to prevent method string caching
dynamic scenario = (Scenario)Activator.CreateInstance(Scenario_Type);
bool result = scenario.InternalInstanceMethod() == "PrivateInstanceMethod";
Console.WriteLine("\nRedirection {0}", result ? "SUCCESS" : "FAILED");
Console.ReadKey();
}
internal string InternalInstanceMethod()
{
return "InternalInstanceMethod";
}
private string PrivateInstanceMethod()
{
return "PrivateInstanceMethod";
}
}
}
This is distilled from a more detailed project I made available on Github (MethodRedirect).
Remark : The code was implemented using .NET Framework 4 and it has not been tested on newer version of .NET.
You can replace a method at runtime by using the ICLRPRofiling Interface.
Call AttachProfiler to attach to the process.
Call SetILFunctionBody to replace the method code.
See this blog for more details.
I know it is not the exact answer to your question, but the usual way to do it is using factories/proxy approach.
First we declare a base type.
public class SimpleClass
{
public virtual DTask<bool> Solve(int n, DEvent<bool> callback)
{
for (int m = 2; m < n - 1; m += 1)
if (m % n == 0)
return false;
return true;
}
}
Then we can declare a derived type (call it proxy).
public class DistributedClass
{
public override DTask<bool> Solve(int n, DEvent<bool> callback)
{
CodeToExecuteBefore();
return base.Slove(n, callback);
}
}
// At runtime
MyClass myInstance;
if (distributed)
myInstance = new DistributedClass();
else
myInstance = new SimpleClass();
The derived type can be also generated at runtime.
public static class Distributeds
{
private static readonly ConcurrentDictionary<Type, Type> pDistributedTypes = new ConcurrentDictionary<Type, Type>();
public Type MakeDistributedType(Type type)
{
Type result;
if (!pDistributedTypes.TryGetValue(type, out result))
{
if (there is at least one method that have [Distributed] attribute)
{
result = create a new dynamic type that inherits the specified type;
}
else
{
result = type;
}
pDistributedTypes[type] = result;
}
return result;
}
public T MakeDistributedInstance<T>()
where T : class
{
Type type = MakeDistributedType(typeof(T));
if (type != null)
{
// Instead of activator you can also register a constructor delegate generated at runtime if performances are important.
return Activator.CreateInstance(type);
}
return null;
}
}
// In your code...
MyClass myclass = Distributeds.MakeDistributedInstance<MyClass>();
myclass.Solve(...);
The only performance loss is during construction of the derived object, the first time is quite slow because it will use a lot of reflection and reflection emit.
All other times, it is the cost of a concurrent table lookup and a constructor.
As said, you can optimize construction using
ConcurrentDictionary<Type, Func<object>>.
have a look into Mono.Cecil:
using Mono.Cecil;
using Mono.Cecil.Inject;
public class Patcher
{
public void Patch()
{
// Load the assembly that contains the hook method
AssemblyDefinition hookAssembly = AssemblyLoader.LoadAssembly("MyHookAssembly.dll");
// Load the assembly
AssemblyDefinition targetAssembly = AssemblyLoader.LoadAssembly("TargetAssembly.dll");
// Get the method definition for the injection definition
MethodDefinition myHook = hookAssembly.MainModule.GetType("HookNamespace.MyHookClass").GetMethod("MyHook");
// Get the method definition for the injection target.
// Note that in this example class Bar is in the global namespace (no namespace), which is why we don't specify the namespace.
MethodDefinition foo = targetAssembly.MainModule.GetType("Bar").GetMethod("Foo");
// Create the injector
InjectionDefinition injector = new InjectionDefinition(foo, myHook, InjectFlags.PassInvokingInstance | InjectFlags.passParametersVal);
// Perform the injection with default settings (inject into the beginning before the first instruction)
injector.Inject();
// More injections or saving the target assembly...
}
}
I have a generic class
class UnmanagedArray<T> where T: struct
{
IntPtr _backing; // unmanaged memory
// some constructors and others methods
// very slow but I don't care
public T this[int index]
{
get
{
int elementSize = Marshal.SizeOf(default(T));
byte* ptr = ((byte*)tab) + index * elementSize;
T result = default(T);
var handle = GCHandle.Alloc(result, GCHandleType.Pinned);
byte* result_ptr = (byte*) handle.AddrOfPinnedObject();
for(int k = 0; k < elementSize; ++k)
{
result_ptr[k] = ptr[k];
}
handle.Free();
return result;
}
}
}
Now I'd like to modify elements in my array.
Something like
myArray[i].prop = 12;
I can't do it because
cannot modify the return value of Array.this[int] because it is
not a variable.
I totally understand the error, since T is a ValueType, therefore immutable (returned by copy).
Is there any way to make a ref return in my indexer property?
PS : T cannot be a reference type. It has to be a ValueType.
There are quite a few questions on here about MemberwiseClone, but I can't find anything covering this exactly.
As I understand it, MemberwiseClone basically just copies an area of memory for an object, dumps it somewhere else and calls it a new instance of that object. Obviously very quick, and for large objects it is the quickest way to make a copy. For small objects with simple constructors, it's quicker to create an instance and copy the individual properties.
Now, I have a tight loop in which I'm using MemberwiseClone. As MemberwiseClone always creates a new instance this leads to a lot of memory allocation, and reallocation which isn't great for performance. I've written a custom copy for a very large object that copies all the properties individually onto an existing object, which overall is marginally quicker than using MemberwiseClone,
I feel that if I could grab the whole chunk of memory and dump it onto the existing object, I'd achieve a decent performance boost as the GC wouldn't have to worry about anything. Hopefully I'll also get rid of this message:
Just to clarify, I want to copy the properties from one existing object to another as quickly as possible. A shallow copy is all I need.
Message 2 DA0023: (Average)% Time in GC = 19.30;
% Time in GC is relatively high. This indication of excessive amount of garbage collection overhead could be impacting the
responsiveness of your application. You can gather .NET memory
allocation data and object lifetime information to understand the
pattern of memory allocation your application uses better.
I have no restrictions on code safety, speed is the aim of the game here.
No comments asking if this is really necessary or talking about premature optimisation please.
Thanks for your help.
Answer
Taking the advise below, of embedding a struct within the object and storing all properties in that. I could then take a copy of that struct to copy all the properties in a single assignment. This yielded more than a 50% speed increase compared to copying field by field, and about a 60% speed increase over creating a new object every time with MemberwiseClone.
If your class is not inheritable, you could store the entire state of your class in an exposed-field structure (expose fields, not properties!). This would require that you prefix all field names with the name of the structure, which would be ugly in the source code, but accessing fields in that structure will be just as fast as would be accessing fields that were directly placed in the enclosing class. Doing this, however, will gain you the ability to copy the structure from one object instance into another using a simple assignment statement.
In general you can't be faster than doing a copy field by field.
This in general. There is a single exception: if your class is blittable (so doesn't contain any reference to non-blittable classes) AND it's declared with the [StructLayout(LayoutKind.Sequential)] (or with the [StructLayout(LayoutKind.Explicit)], but it's more complex to test) then you can pin it
GCHandle handle = GCHandle.Alloc(refToYourClass, GCHandleType.Pinned);
from there a new world opens... You can directly copy byte-by-byte the content of your class with some unsafe (both as unsafe the keyboard and unsafe as "running-with-scissors") code.
BUT for this to work, your class MUST be blittable!
The fastest method is UnsafeCopy8 that copies blocks of 8 bytes (both at 32bits and at 64bits). The fastest PInvoke method is memcpy.
[StructLayout(LayoutKind.Sequential)]
class MyClass
{
public int A { get; set; }
public int B { get; set; }
public int C { get; set; }
public int D { get; set; }
public int E { get; set; }
public int F { get; set; }
public int G { get; set; }
public byte H { get; set; }
}
class Program
{
[DllImport("msvcrt.dll", CallingConvention = CallingConvention.Cdecl, SetLastError = false)]
static extern IntPtr memcpy(IntPtr dest, IntPtr src, UIntPtr count);
[DllImport("kernel32.dll", SetLastError = false)]
static extern void CopyMemory(IntPtr dest, IntPtr src, UIntPtr count);
static void Main()
{
MyClass mc = new MyClass { A = 1, B = 2, C = 3, D = 4, E = 5, F = 6, G = 7, H = 8 };
MyClass mc2 = new MyClass();
int size = Marshal.SizeOf(typeof(MyClass));
var gc = GCHandle.Alloc(mc, GCHandleType.Pinned);
var gc2 = GCHandle.Alloc(mc2, GCHandleType.Pinned);
IntPtr dest = gc2.AddrOfPinnedObject();
IntPtr src = gc.AddrOfPinnedObject();
// Pre-caching
memcpy(dest, src, (UIntPtr)size);
CopyMemory(dest, src, (UIntPtr)size);
UnsafeCopy(dest, src, size);
UnsafeCopy8(dest, src, size);
int cycles = 10000000;
Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High;
var sw1 = Stopwatch.StartNew();
for (int i = 0; i < cycles; i++)
{
memcpy(dest, src, (UIntPtr)size);
}
sw1.Stop();
var sw2 = Stopwatch.StartNew();
for (int i = 0; i < cycles; i++)
{
CopyMemory(dest, src, (UIntPtr)size);
}
sw2.Stop();
var sw3 = Stopwatch.StartNew();
for (int i = 0; i < cycles; i++)
{
UnsafeCopy(dest, src, size);
}
sw3.Stop();
var sw4 = Stopwatch.StartNew();
for (int i = 0; i < cycles; i++)
{
UnsafeCopy8(dest, src, size);
}
sw4.Stop();
gc.Free();
gc2.Free();
Console.WriteLine("IntPtr.Size: {0}", IntPtr.Size);
Console.WriteLine("memcpy: {0}", sw1.ElapsedTicks);
Console.WriteLine("CopyMemory: {0}", sw2.ElapsedTicks);
Console.WriteLine("UnsafeCopy: {0}", sw3.ElapsedTicks);
Console.WriteLine("UnsafeCopy8: {0}", sw4.ElapsedTicks);
Console.ReadKey();
}
static unsafe void UnsafeCopy(IntPtr dest, IntPtr src, int size)
{
while (size >= sizeof(int))
{
*((int*)dest) = *((int*)src);
dest = (IntPtr)(((byte*)dest) + sizeof(int));
src = (IntPtr)(((byte*)src) + sizeof(int));
size -= sizeof(int);
}
if (size >= sizeof(short))
{
*((short*)dest) = *((short*)src);
dest = (IntPtr)(((byte*)dest) + sizeof(short));
src = (IntPtr)(((byte*)src) + sizeof(short));
size -= sizeof(short);
}
if (size == sizeof(byte))
{
*((byte*)dest) = *((byte*)src);
}
}
static unsafe void UnsafeCopy8(IntPtr dest, IntPtr src, int size)
{
while (size >= sizeof(long))
{
*((long*)dest) = *((long*)src);
dest = (IntPtr)(((byte*)dest) + sizeof(long));
src = (IntPtr)(((byte*)src) + sizeof(long));
size -= sizeof(long);
}
if (size >= sizeof(int))
{
*((int*)dest) = *((int*)src);
dest = (IntPtr)(((byte*)dest) + sizeof(int));
src = (IntPtr)(((byte*)src) + sizeof(int));
size -= sizeof(int);
}
if (size >= sizeof(short))
{
*((short*)dest) = *((short*)src);
dest = (IntPtr)(((byte*)dest) + sizeof(short));
src = (IntPtr)(((byte*)src) + sizeof(short));
size -= sizeof(short);
}
if (size == sizeof(byte))
{
*((byte*)dest) = *((byte*)src);
}
}
}
I'm working in C# with a Borland C API that uses a lot of byte pointers for strings. I've been faced with the need to pass some C# strings as (short lived) byte*.
It would be my natural assumption that a const object would not be allocated on the heap, but would be stored directly in program memory, but I've been unable to verify this in any documentation.
Here's an example of what I've done in order to generate a pointer to a constant string. This does work as intended in testing, I'm just not sure if it's really safe, or it's only working via luck.
private const string pinnedStringGetWeight = "getWeight";
unsafe public static byte* ExampleReturnWeightPtr(int serial)
{
fixed (byte* pGetWeight = ASCIIEncoding.ASCII.GetBytes(pinnedStringGetWeight))
return pGetWeight;
}
Is this const really pinned, or is there a chance it could be moved?
#Kragen:
Here is the import:
[DllImport("sidekick.dll", CallingConvention = CallingConvention.Winapi)]
public static extern int getValueByFunctionFromObject(int serial, int function, byte* debugCallString);
This is the actual function. Yes, it actually requires a static function pointer:
private const int FUNC_GetWeight = 0x004243D0;
private const string pinnedStringGetWeight = "getWeight";
unsafe public static int getWeight(int serial)
{
fixed (byte* pGetWeight = ASCIIEncoding.ASCII.GetBytes(pinnedStringGetWeight))
return Core.getValueByFunctionFromObject(serial, FUNC_GetWeight, pGetWeight);
}
Following is another method that I used when mocking my API, using a static struct, which I also hoped was pinned. I was hoping to find a way to simplify this.
public byte* getObjVarString(int serial, byte* varName)
{
string varname = StringPointerUtils.GetAsciiString(varName);
string value = MockObjVarAttachments.GetString(serial, varname);
if (value == null)
return null;
return bytePtrFactory.MakePointerToTempString(value);
}
static UnsafeBytePointerFactoryStruct bytePtrFactory = new UnsafeBytePointerFactoryStruct();
private unsafe struct UnsafeBytePointerFactoryStruct
{
fixed byte _InvalidScriptClass[255];
fixed byte _ItemNotFound[255];
fixed byte _MiscBuffer[255];
public byte* InvalidScriptClass
{
get
{
fixed (byte* p = _InvalidScriptClass)
{
CopyNullString(p, "Failed to get script class");
return p;
}
}
}
public byte* ItemNotFound
{
get
{
fixed (byte* p = _ItemNotFound)
{
CopyNullString(p, "Item not found");
return p;
}
}
}
public byte* MakePointerToTempString(string text)
{
fixed (byte* p = _ItemNotFound)
{
CopyNullString(p, text);
return p;
}
}
private static void CopyNullString(byte* ptrDest, string text)
{
byte[] textBytes = ASCIIEncoding.ASCII.GetBytes(text);
fixed (byte* p = textBytes)
{
int i = 0;
while (*(p + i) != 0 && i < 254 && i < textBytes.Length)
{
*(ptrDest + i) = *(p + i);
i++;
}
*(ptrDest + i) = 0;
}
}
}
How constants are allocated shouldn't matter in this case, because ASCIIEncoding.ASCII.GetBytes() returns a new byte array (it cannot return the constant's internal array, since it is encoded differently (edit: there is hopefully no way to get a pointer to a string's internal array, since strings are immutable)). However, the guarantee that the GC won't touch the array only lasts as long as the fixed scope does - in other words, when the function returns, the memory is no longer pinned.
Based on Kragans comment, I looked into the proper way to marshall my string to a byte pointer and am now using the following for the first example I used in my question:
[DllImport("sidekick.dll", CallingConvention = CallingConvention.Winapi)]
public static extern int getValueByFunctionFromObject(int serial, int function, [MarshalAs(UnmanagedType.LPStr)]string debugCallString);
I want to convert a byte* to a byte[], but I also want to have a reusable function to do this:
public unsafe static T[] Create<T>(T* ptr, int length)
{
T[] array = new T[length];
for (int i = 0; i < length; i++)
array[i] = ptr[i];
return array;
}
Unfortunately I get a compiler error because T might be a ".NET managed type" and we can't have pointers to those. Even more frustrating is that there is no generic type constraint which can restrict T to "unmanaged types". Is there a built-in .NET function to do this? Any ideas?
The method that could match what you are trying to do is Marshal.Copy, but it does not take the appropriate parameters to make a generic method.
Although there it is not possible to write a generic method with generic constraints that could describe what is possible, not every type can be allowed to be copied using an "unsafe" way. There are some exceptions; classes are one of these.
Here is a sample code:
public unsafe static T[] Create<T>(void* source, int length)
{
var type = typeof(T);
var sizeInBytes = Marshal.SizeOf(typeof(T));
T[] output = new T[length];
if (type.IsPrimitive)
{
// Make sure the array won't be moved around by the GC
var handle = GCHandle.Alloc(output, GCHandleType.Pinned);
var destination = (byte*)handle.AddrOfPinnedObject().ToPointer();
var byteLength = length * sizeInBytes;
// There are faster ways to do this, particularly by using wider types or by
// handling special lengths.
for (int i = 0; i < byteLength; i++)
destination[i] = ((byte*)source)[i];
handle.Free();
}
else if (type.IsValueType)
{
if (!type.IsLayoutSequential && !type.IsExplicitLayout)
{
throw new InvalidOperationException(string.Format("{0} does not define a StructLayout attribute", type));
}
IntPtr sourcePtr = new IntPtr(source);
for (int i = 0; i < length; i++)
{
IntPtr p = new IntPtr((byte*)source + i * sizeInBytes);
output[i] = (T)System.Runtime.InteropServices.Marshal.PtrToStructure(p, typeof(T));
}
}
else
{
throw new InvalidOperationException(string.Format("{0} is not supported", type));
}
return output;
}
unsafe static void Main(string[] args)
{
var arrayDouble = Enumerable.Range(1, 1024)
.Select(i => (double)i)
.ToArray();
fixed (double* p = arrayDouble)
{
var array2 = Create<double>(p, arrayDouble.Length);
Assert.AreEqual(arrayDouble, array2);
}
var arrayPoint = Enumerable.Range(1, 1024)
.Select(i => new Point(i, i * 2 + 1))
.ToArray();
fixed (Point* p = arrayPoint)
{
var array2 = Create<Point>(p, arrayPoint.Length);
Assert.AreEqual(arrayPoint, array2);
}
}
The method can be generic, but it cannot take a pointer of a generic type. This is not an issue since pointers covariance is helping, but this has the unfortunate effect of preventing an implicit resolution of the generic argument type. You then have to specify MakeArray explicitly.
I've added a special case for the structures, where it is best to have types that specify a struct layout. This might not be an issue in your case, but if the pointer data is coming from native C or C++ code, specifying a layout kind is important (The CLR might choose to reorder fields to have a better memory alignment).
But if the pointer is coming exclusively from data generated by managed code, then you can remove the check.
Also, if the performance is an issue, there are better algorithms to copy the data than doing it byte by byte. (See the countless implementations of memcpy for reference)
Seems that the question becomes: How to specify a generic Type to be a simple type.
unsafe void Foo<T>() : where T : struct
{
T* p;
}
Gives the error:
Cannot take the address of, get the size of, or declare a pointer to a managed type ('T')
How about this?
static unsafe T[] MakeArray<T>(void* t, int length, int tSizeInBytes) where T:struct
{
T[] result = new T[length];
for (int i = 0; i < length; i++)
{
IntPtr p = new IntPtr((byte*)t + (i * tSizeInBytes));
result[i] = (T)System.Runtime.InteropServices.Marshal.PtrToStructure(p, typeof(T));
}
return result;
}
We can't use sizeof(T) here, but the caller can do something like
byte[] b = MakeArray<byte>(pBytes, lenBytes, sizeof(byte));
I have no idea whatsoever if the following would work, but it might (at least it compiles :):
public unsafe static T[] Create<T>(void* ptr, int length) where T : struct
{
T[] array = new T[length];
for (int i = 0; i < length; i++)
{
array[i] = (T)Marshal.PtrToStructure(new IntPtr(ptr), typeof(T));
}
return array;
}
The key is to use Marshal.PtrToStructure to convert to the correct type.
As of C# 7.3 this is possible as follows:
public unsafe static T[] Create<T>(T* ptr, int length) where T: unmanaged
{
T[] array = new T[length];
for (int i = 0; i < length; i++)
array[i] = ptr[i];
return array;
}