Develop Reference

Delegate on instance method passed by P/Invoke

To my surprise I have discovered a powerful feature today. Since it looks too good to be true I want to make sure that it is not just working due to some weird coincidence.
I have always thought that when my p/invoke (to c/c++ library) call expects a (callback) function pointer, I would have to pass a delegate on a static c# function. For example in the following I would always reference a delegate of KINSysFn to a static function of that signature.
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate int KINSysFn(IntPtr uu, IntPtr fval, IntPtr user_data );
and call my P/Invoke with this delegate argument:
[DllImport("some.dll", EntryPoint = "KINInit", ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)]
public static extern int KINInit(IntPtr kinmem, KINSysFn func, IntPtr tmpl);
But now I just tried and passed a delegate on an instance method and it worked also! For example:
public class MySystemFunctor
{
double c = 3.0;
public int SystemFunction(IntPtr u, IntPtr v, IntPtr userData) {}
}
// ...
var myFunctor = new MySystemFunctor();
KINInit(kinmem, myFunctor.SystemFunction, IntPtr.Zero);
Of course, I understand that inside managed code there is no technical problem at all with packaging a "this" object together with an instance method to form the respective delegate.
But what surprises me about it, is the fact that the "this" object of MySystemFunctor.SystemFunction finds its way also to the native dll, which only accepts a static function and does not incorporate any facility for a "this" object or packaging it together with the function.
Does this mean that any such delegate is translated (marshalled?) individually to a static function where the reference to the respective "this" object is somehow hard coded inside the function definition? How else could be distinguished between the different delegate instances, for example if I have
var myFunctor01 = new MySystemFunctor();
// ...
var myFunctor99 = new MySystemFunctor();
KINInit(kinmem, myFunctor01.SystemFunction, IntPtr.Zero);
// ...
KINInit(kinmem, myFunctor99.SystemFunction, IntPtr.Zero);
These can't all point to the same function. And what if I create an indefinite number of MySystemFunctor objects dynamically? Is every such delegate "unrolled"/compiled to its own static function definition at runtime?

Does this mean that any such delegate is translated (marshalled?) individually to a static function...
Yes, you guessed at this correctly. Not exactly a "static function", there is a mountain of code inside the CLR that performs this magic. It auto-generates machine code for a thunk that adapts the call from native code to managed code. The native code gets a function pointer to that thunk. The argument values may have to be converted, a standard pinvoke marshaller duty. And always shuffled around to match the call to the managed method. Digging up the stored delegate's Target property to provide this is part of that. And it jiggers the stack frame, tying a link to the previous managed frame, so the GC can see that it again needs to look for object roots.
There is however one nasty little detail that gets just about everybody in trouble. These thunks are automatically cleaned-up again when the callback is not necessary anymore. The CLR gets no help from the native code to determine this, it happens when the delegate object gets garbage-collected. Maybe you smell the rat, what determines in your program when that happens?
var myFunctor = new MySystemFunctor();
That is a local variable of a method. It is not going to survive for very long, the next collection will destroy it. Bad news if the native code keeps making callbacks through the thunk, it won't be around anymore and that's a hard crash. Not so easy to see when you are experimenting with code since it takes a while.
You have to ensure this can't happen. Storing the delegate objects in your class might work, but then you have to make sure your class object survives long enough. Whatever it takes, no guess from the snippet. It tends to solve itself when you also ensure that you unregister these callbacks again since that requires storing the object reference for use later. You can also store them in a static variable or use GCHandle.Alloc(), but that of course loses the benefit of having an instance callback in a hurry. Feel good about having this done correctly by testing it, call GC.Collect() in the caller.
Worth noting is that you did it right by new-ing the delegate explicitly. C# syntax sugar does not require that, makes it harder to get this right. If the callbacks only occur while you make the pinvoke call into the native code, not uncommon (like EnumWindows), then you don't have to worry about it since the pinvoke marshaller ensures the delegate object stays referenced.

For the records: I have walked right into the trap, Hans Passant has mentioned. Forced garbage collection has led to a null reference exception because the delegate was transient:
KINInit(kinmem, myFunctor.SystemFunction, IntPtr.Zero);
// BTW: same with:
// KINInit(kinmem, new KINSysFn(myFunctor.SystemFunction), IntPtr.Zero);
GC.Collect();
GC.WaitForPendingFinalizers();
KINSol(/*...*); // BAAM! NullReferenceException
Luckily I had already wrapped the critical two P/Invokes, KINInit (which sets the callback delegate) and KINSolve (which actually uses the callback) into a dedicated managed class. The solution was, as already discussed, to keep the delegate referenced by a class member:
// ksf is a class member of delegate type KINSysFn that keeps ref to delegate instance
ksf = new KINSysFn(myFunctor.SystemFunction);
KINInit(kinmem, ksf, IntPtr.Zero);
GC.Collect();
GC.WaitForPendingFinalizers();
KINSol(/*...*);
Thanks again, Hans, I'd never have noticed this flaw because it works as long as no GC happens!

C# Interop - Releasing memory allocated in unmanaged code

I'm calling the following VC++ method
__declspec(dllexport) unsigned char* Get_Version_String()
from C# as follows:
internal static class NativeMethods
{
[DllImport("my.dll"),
CharSet = CharSet.Ansi, BestFitMapping = false, ThrowOnUnmappableChar = true,
CallingConvention = CallingConvention.Cdecl)]
internal static extern string Get_Version_String();
}
The above code is in a library which targets .NET 3.5. When I call this from a 3.5 assembly, it works fine; when calling it from a 4.5 assembly, however, it results in
0xC0000374: A heap has been corrupted
After reading this question, I changed my method calls as follows:
[DllImport("my.dll",
EntryPoint = "Get_Version_String",
CharSet = CharSet.Ansi, BestFitMapping = false, ThrowOnUnmappableChar = true,
CallingConvention = CallingConvention.Cdecl)]
private static extern IntPtr Get_Version_String_PInvoke();
internal static string Get_Version_String()
{
IntPtr ptr = Get_Version_String_PInvoke();
string versionString = Marshal.PtrToStringAnsi(ptr);
return versionString;
}
This works as expected, but the answer from Hans Passant comes with a warning:
The workaround you found is the correct one, the marshaller isn't going to try to release the memory for an IntPtr. Do note that this will only actually come to a good end if the C code returns a const char* that doesn't need to be released. You have a permanent memory leak if that's not the case.
Since the C++ method does not return const, I'm going on the assumption that the workaround for my specific function will result in a memory leak.
I cannot change the original method so I found this other question which discusses how to free memory from manged code. However, calling either Marshal.FreeHGlobal(ptr) or Marshal.FreeCoTaskMem(ptr) also throw 0xC0000374: A heap has been corrupted.
Can anyone
a) confirm that such a method will indeed suffer from a memory leak, and
b) if so, suggest how to free the memory from the pointer in managed code?
The C++ method body is, simplified, as follows:
unsigned char versionString[50];
__declspec(dllexport) unsigned char* Get_Version_String()
{
strcpy((char *) versionString, "Key1:[xx],Key2:[xx],Key3:[xx],Key4:[xx]");
// string manipulation
return versionString;
}
Thanks in advance, and sorry if this is trivial; I'm neither a C++ nor an Interop expert.

Can anyone confirm that such a method will indeed suffer from a memory leak
Only you can do that, the missing const keyword is not a guarantee that the native code does not in fact return a literal. Pervasive mistake in C code btw. Write a little test program that calls the function a hundred million times. If you don't see the memory usage explode with Task Manager then you don't have a problem.
if so, suggest how to free the memory from the pointer in managed code?
You just can't, it must be the native code itself that calls free(). So that it uses the correct heap, the one that was created by the C runtime library used by that code. Underlying winapi call is HeapCreate(), you don't have the heap handle. Technically it is discoverable with GetProcessHeaps() but you just don't know which one is the "right" one. Starting with VS2012, the CRT uses GetProcessHeap() instead of HeapCreate(), now Marshal.FreeHGlobal() can work. But you know that this code doesn't, you'll have to ask the author or vendor for an update. As long as you do that, ask him for a more usable flavor of this function, it should take a char* as an argument instead.
A more constructive approach is to just take the memory leak in stride. Simply call the function once, the version number is not going to change while your program is running. So store it in a static variable. Losing ~80 bytes of address space is not a problem you can ever notice, the OS automatically cleans up when your program terminates.

Using Managed Class Method as Callback

I'm currently working on a little Midi Project and just now I'm writing a little "SharpMidi" Library in C++/Cli to make it easier for me to use native Midi functions from Managed code.
I'm now stuck on the midiInOpen Function:
http://msdn.microsoft.com/en-us/library/windows/desktop/dd798458%28v=vs.85%29.aspx
As you can see from the documentation, this function requires an unmanaged function as Callback and optional instance data.
I managed to convert a Managed Function Pointer to an unmanaged Function Pointer using
System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate
but I now have problems converting the this Pointer from my managed class to an unmanaged Pointer, I don't even know if its possibly they way I want it to work.
HMIDIIN _midiInDevice;
void MidiInProcNative(HMIDIIN midiInDevice, UINT msg, DWORD_PTR dwInstance, DWORD_PTR dwParam1, DWORD_PTR dwParam2);
delegate void MidiInProc(HMIDIIN midiInDevice, UINT msg, DWORD_PTR dwInstance, DWORD_PTR dwParam1, DWORD_PTR dwParam2);
MidiInProc^ _midiInProc;
...
_midiInProc = gcnew MidiInDevice::MidiInProc(this, &MidiInDevice::MidiInProcNative);
cli::pin_ptr<HMIDIIN> midiInDevPtr = &_midiInDevice;
pin_ptr<???> thisPtr = this; // This is the line I'm struggling with
MMRESULT result = midiInOpen(midiInDevPtr, devId, (DWORD_PTR)System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate(_midiInProc).ToPointer(), thisPtr, CALLBACK_FUNCTION);
As you can see I have experimented with pin_ptrs but I don't think this will work as the this pointer can't be used to initialize any pin_ptr.
I hope I could make clear what I want to try to achieve and maybe somebody can help me out.
Regards,
Xaser

I now have problems converting the this Pointer from my managed class to an unmanaged Pointer
Just don't, there's no need for it. The dwCallbackInstance argument of miniInOpen() is a convenience argument for native C++. You get it back in the static callback function, let's you call an instance method of a C++ class.
But delegates are far more powerful than member function pointers, they already encapsulate this. A delegate constructor takes two arguments, the target object and the target method. So that it can automatically call an instance method. You already took advantage of that, you indeed passed this as the first argument. So you can be sure that your MidiInProcNative() instance method uses the proper object reference.
Just pass nullptr for the argument.

marshalling C and C#

I have the following marshalling code in my project. I have few questions on this.
[DllImport=(Core.dll, SetLastError=true, EntryPoint="CoreCreate", CharSet="CharSet.Ansi", CallingConvention="CallingConvention.Cdecl")]
internal static extern uint CoreCreate(ref IntPtr core);
Why 'internal static extern' is required? Is this compulsory? Why this is used?
What is SetLastError?
[StructLayout(LayoutKind.Sequential,CharSet=CharSet.Ansi)]
internal struct Channel
{
internal byte LogicalChannel;
}
Why LayoutKind.Sequential?

Why 'internal static extern' is required?
The internal modifier just set visibility of your method. It's not required to be internal so you can declare the method private or public as you need and as you would do with any other standard method.
The static modifier is required because it's not an instance method and that method doesn't know any class (it hasn't a this pointer).
Finally extern is required to inform the compiler that the method isn't implemented here but in another place (and you'll specify where using attributes). Evey extern method must be declared static too (because it's a simple function call without any knowledge about objects).
What is SetLastError?
It indicates that method may change the thread's last-error code value. See the GetLastError() function for details about this. If the called function will change this value then it's a good thing to set SetLastError to true, from MSDN:
The runtime marshaler calls GetLastError and caches the value returned to prevent it from being overwritten by other API calls. You can retrieve the error code by calling GetLastWin32Error.
In short it saves the value returned by GetLastError() to an internal cache so any other call to system API (even internal to others framework functions) won't overwrite that value.
Why LayoutKind.Sequential?
Class layout in .NET isn't required to be sequential in memory (sequential = if A is declared before B then memory layout has A before B). This is not true in C where the declaration order matters (declaration is used by the compiler to understand the layout, in memory, of raw data). If you have to interop with C functions then you have to be sure about the layout of the data you pass them. This is how LayoutKind.Sequential works: it instructs the compiler to respect the declaration sequential order for data in the struct. This is not the only option to interop with unmanaged world, you can even explicitly set the offset (from structure beginning) of each field (see LayoutKind.Explicit).

This is not an answer, just a few comments:
"internal static" is one thing and "extern" is another thing needed when calling external dll's.
SetLastError or GetLastError is methods we used a lot in the "old" days to get error messages from windows about the latest handling.
LayoutKind.Sequential is a way to inform the compiler to layout the struct in a specified way - you may need to do this if marchalling to other systems.

Calling _msize() via PInvoke from C#

I'm writing a C# library where the calling app will pass in a large amount of contiguous, unmanaged memory. This calling app can be either from .Net or Visual C++ (it will go through an intermediate C++/CLI library before calling my library if from C++). It would be useful to validate that there is sufficient memory, so I decided to call the _msize() function. Unfortunately, _msize always seems to give me the wrong size back.
I went back and modified my allocation routine in my sample app and then immediately call _msize. Here is my code snipet:
public unsafe class MyMemory
{
/// <returns></returns>
[DllImport("msvcrt.dll", SetLastError = true)]
public static extern int _msize(IntPtr handle);
public static IntPtr MyAlloc(int size)
{
IntPtr retVal = Marshal.AllocHGlobal(size);
...
int memSize = MyMemory._msize(retVal);
if (memSize < size)
{
...
}
return retVal;
}
When I pass in the size 199229440, I get back memSize of 199178885. I've seen similar results for different numbers. It is less than 0.01% off, which I would totally understand if it was over, but the fact is it is under, meaning _msize thinks the allocated memory is less than what was asked for. Anyone have any clue why this is? And any recommendations on what I should do instead would be appreciated as well.

P/Invoke the LocalSize function instead.
_msize is for determining the size of a block allocated with malloc (and its friends). AllocHGlobal is a wrapper around GlobalAlloc or LocalAlloc (depending on what reference you believe; but I think the two are equivalent), and you want the LocalSize function to determine the size of the block that actually returned. So far as I can tell, Marshal doesn't contain a wrapper for LocalSize, but you can call it using P/Invoke.
So it seems like it's only by sheer good luck that _msize is returning anything useful for you at all. Perhaps malloc uses GlobalAlloc (or LocalAlloc), either always or just when asked for large blocks, and requests a bit of extra space for bookkeeping; in which case _msize would be trying to compensate for that.