Develop Reference

How to get IntPtr of c# unsafe struct to free its memory initialized in c++ dll

Please, advise me on how to free memory from the created unsafe C# struct(s) using some standard C# toolset or how get I get the IntPtr of those objects to use the default provided custom C++ library?
The problem details (UPD):
I create the C# unsafe struct and pass it to C++ dll that I can't change TH_ExtractBimTemplate(ref createdStruct). The dll updates its content. And in addition to that, the dll saves the reference to the struct object inside of it in an internal list. Here is the struct definition:
public unsafe struct BIM_Template
{
public ushort ImageSizeX;
public ushort ImageSizeY;
public fixed uint Mint[MAX_MINUTIAE_SIZE];
public uint Reserve;
}
After the API call returns, I use the data in the struct populated by the C++ dll in my C# code.
I need to free the memory used by the struct and C++. The dll has an API to do this, TH_FreeMemory.
For step #3 I need to pass the object reference (I presume IntPtr) to C++ library or I need to Free the memory it was occupying with some C# instrument.
The Garbage Collection doesn't work even if there are no reference in my C# code to the object, I assume, because of the struct object reference is stored in the list in the dll that I don't control. When TH_FreeMemory on the struct object is called, it removes it from the list and release the memory with delete, but I can't generate the C# IntPtr to pass it to the TH_FreeMemory for that...
How can I do that - release the memory with just C# tools, ignoring the list dll or generate the IntPtr for this struct?
The implementation details:
This is how I create and initialize it:
BIM_Template template = default;
var template = TH_ExtractBimTemplate(ref template)
Where TH_ExtractBimTemplate(...) is the method that is pulled from the C++ dll in this way:
[DllImport("TemplateHelper.dll")]
private static extern int TH_ExtractBimTemplate(
ref BIM_Template out_template);
The TemplateHelper.dll provides me a C++ method to release the memory - TH_FreeMemory:
int WINAPI TH_FreeMemory( void *memblock, BIM_MemBlockType memblock_type )
This is how I plug it into C#
[DllImport("TemplateHelper.dll")]
private static extern int TH_FreeMemory(
IntPtr memblock, BIM_MemBlockType memblock_type
);
Current solutions I tried:
I need IntPtr of the created unsafe struct (the template variable) but I don't know how to get it, to free allocated memory with TH_FreeMemory dll method
I have tried to create a bunch of these unsafe struct type templates in the method that create them and return void, leaving no references to the created templates, but the garbage collector doesn't free memory. I assume that is, because of the object type is unsafe struct. At least, I've waited for 40 minutes and memory wasn't released.
I have tried the approach below to release the memory with Marshal.FreeHGlobal(IntPtr ptr) but that doesn't reduce the amount of unmanaged memory used by the app as well here is the code:
[Fact]
public void TestDisposalIsCalledOnOutOfContext()
{
// Create templates
var templates = LoadTemplates(m, Output);
// Release the allocated memory using Marshal
templates.ForEach( t => MarshaFreeMemory(ref t));
// Wait
Thread.Sleep(new TimeSpan(hours: 1, 0, 0));
}
private static IntPtr MarshaFreeMemory(ref BIM_Template? template)
{
IntPtr ptr;
ptr = Marshal.AllocHGlobal(Marshal.SizeOf<BIM_Template>());
Marshal.StructureToPtr(template, ptr, true);
Marshal.FreeHGlobal(ptr);
return ptr;
}
private static List<BIM_Template?> LoadTemplates()
{
var templates = new List<BIM_Template?>();
for (int j = 0; j < 1000; j++) templates.Add(LoadTemplate());
return templates;
}
public static BIM_Template LoadTemplate()
{
BIM_Template template = default;
var template = TH_ExtractBimTemplate(ref template);
return template;
}
So, the question is how can I prevent these memory leaks and free memory from the created templates using some standard C# toolset or how get I get the IntPtr of the template object to use the library?

You don’t need to call TH_FreeMemory. Your structure doesn’t have any pointers inside. In C++, the Mint field should be std::array<uint32_t, MAX_MINUTIAE_SIZE> or an equivalent.
Your template local variable is on the stack, it does not use any heap memory, and the stack memory will be freed automatically just before the LoadTemplate() method returns.
if you wanna free the memory used by the list in TestDisposalIsCalledOnOutOfContext, do this:
templates = null;
// Collect all generations of memory
GC.Collect();

P/Invoke passing struct longer than native code expects

If I have a native code that expects a structure with two fields:
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern int my_method(ref MyStruct myStruct);
// this is what native code expects
public struct MyStruct {
IntPtr First;
IntPtr Second;
}
but instead I pass another struct to it, will it work or not - by design or by accident?
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern int my_method(ref MyLongerStruct myLongerStruct);
// this is what I want to pass to it
public struct MyLongerStruct {
IntPtr First;
IntPtr Second;
object ObjectPointer;
}
Will the object reference added to the end of the struct at C# side somehow affect P/Invoke call?

I shouldn't work. And even more, you need to add and properly set StructLayoutAttribute to the structure, as it explained here
I think, the result should be like this:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct {
IntPtr First;
IntPtr Second;
}

If the total difference in structure is fields added to the end, and you use StructLayout to prevent the compiler from optimizing the memory layout of your struct (as Alex Butenko suggests), then it's unlikely that there will be any negative side effects apart from a slight speed hit.
When you pass a managed struct to an external function via P/Invoke (using the DllImport attribute) there is a marshaling phase that converts your struct to a compatible format for the target. For ref and out parameters the temporary is converted back when the invoked function returns, copying the values back to your struct instance. All of this is abstracted away, although exactly how the marshaling is performed for each member can be tweaked with the right attributes.
This is how the .NET framework handles strings in P/Invoke. Since it can't just send a string instance pointer to an API function that is expecting a char * (the two are nothing alike) there has to be some translation.
The fun part is that the marshaling code doesn't know anything about what the target is expecting other than what you tell it at the C# end, so if you are sending an extended version of the structure it will do the whole thing. At the other end the native code will get a pointer to a memory block containing the information it's expecting, and it won't have any way to tell that there is more after the end of the structure.
Apart from that, no problem... as long as you're passing by reference and not by value. Passing structs by value is something that should raise big red stop signs all over your brain. Don't do it, it's evil.

Managing unmanaged string across C++ / C# boundary with P/Invoke

I have the following struct declared (C++):
struct NativeOperationResult {
const INTEROP_BOOL Success; // INTEROP_BOOL = char
const char16_t* const ErrorMessage;
NativeOperationResult(const NativeOperationResult& c);
/* various constructors, omitted for brevity */
};
Now, I have an exported function definition elsewhere:
extern "C" __declspec(dllexport) NativeOperationResult ReturnFailureWithMessage() {
return { INTEROP_BOOL_FALSE, "Test" };
}
My expectation is to be calling ReturnFailureWithMessage (a test method in case you were wondering) from C# via P/Invoke. In the NativeOperationResult constructor, it takes a copy of "Test" and puts it in ErrorMessage.
The NativeOperationResult has ownership of the char16_t* so I need to delete it when the struct is destroyed. That's no problem, but I don't want to delete the memory before the .NET CLR has a chance to copy the string in to the managed heap.
Frankly I'm a bit fuzzy on where to delete that memory. What I think is that the C++ compiler will make a copy of my struct (or just move it) and then the CLR will use that copy... Which means I should delete the native memory from .NET with Marshal.FreeHGlobal.
Is that correct?

No, that is not correct. You need to differentiate between two cases:
1) You didn't do any allocations on C++ side. This is the case you are talking about right now.
2) You did do allocations on C++ side, you need to take care of deallocation.
Thus to answer your question: no, your example does not need any "deletion" of memory, since no-one has allocated the memory explictly.
Second case is a bit trickier. If you do memory allocations on C++ side with new char16_t[blah], you need to release the memory with delete[] nativeOperationResult.ErrorMessage. This is not possible to do on C# side. The memory can be allocated using different allocators(such as; malloc, new) and C# does not know how to deal with these pointers.
You need to add a new flag to NativeOperationResult, such as DeletionRequired, and export new function from unmanaged side: FreeNativeOperationResultIfNeeded(..). There is longer discussion here.
You can avoid all this non-sense with C++ strings. They work magically, and no deletion is required.
struct NativeOperationResult {
const INTEROP_BOOL Success; // INTEROP_BOOL = char
const string const ErrorMessage;
NativeOperationResult(const NativeOperationResult& c);
/* various constructors, omitted for brevity */
};

Instead of returning NativeOperationResult, you can make it an out parameter and expect the calling method to pass it the right amount of memory. That way the .Net can allocate memory and then clean it up after the pinvoke is done. But you will have to figure out a way to inform .Net the amount of memory it is expected to allocate.
extern "C" __declspec(dllexport) void ReturnFailureWithMessage(NativeOperationResult* result)

Why can't I do Marshal.SizeOf() for this C# struct?

I try to call the code int size = Marshal.SizeOf(typeof(MyStruct)) but it throws the following exception:
Type 'MyStruct' cannot be marshaled as an unmanaged structure; no meaningful size or offset can be computed.
My struct is as follows:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.U4)]
public UInt32 version;
[MarshalAs(UnmanagedType.FunctionPtr)]
public IntPtr Start;
[MarshalAs(UnmanagedType.FunctionPtr)]
public IntPtr Stop;
// And a bunch more IntPtr, all declared the same way.
}
The struct is supposed to be passed to C-land, where the C code will use its contents as function pointers. I can't see how computing a size would fail, can anyone help?

UnmanagedType.FunctionPtr requires the field to be a delegate type. It will be a function pointer on the C-side after the structure is marshaled. Using [MarshalAs] is superfluous, a delegate already gets marshaled like that. So, roughly:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.U4)]
public UInt32 version;
public Action Start;
public Func<bool> Stop;
// etc..
}
Change the delegate types to match the function signature of the corresponding C function pointer. You often have to declare your own delegate type so you can give it the [UnmanagedFunctionPointer] attribute to match the calling convention of the C function. Usually CallingConvention.Cdecl, not the default of Stdcall.
You have to be very careful when you initialize a structure like this. The delegate objects you create and assign to the fields must be referenced elsewhere to prevent them from getting garbage collected. Either by storing them in a class object that's guaranteed to live as long as the C code can make calls, by storing them in a static variable or by explicitly adding a reference with GCHandle.Alloc()
Lots of ways to shoot your foot, good luck with it :)

C#: Pointer to the struct inside the struct

I am trying to use marshalling in C#. In C++ I have a this struct:
struct aiScene
{
unsigned int mFlags;
C_STRUCT aiNode* mRootNode;
unsigned int mNumMeshes;
C_STRUCT aiMesh** mMeshes;
unsigned int mNumMaterials;
C_STRUCT aiMaterial** mMaterials;
unsigned int mNumAnimations;
C_STRUCT aiAnimation** mAnimations;
unsigned int mNumTextures;
C_STRUCT aiTexture** mTextures;
unsigned int mNumLights;
C_STRUCT aiLight** mLights;
unsigned int mNumCameras;
C_STRUCT aiCamera** mCameras;
}
So, C# eqvivalent is:
[StructLayout(LayoutKind.Sequential)]
public struct aiScene
{
public uint mFlags;
public unsafe aiNode* mRootNode;
public uint mNumMeshes;
public unsafe aiMesh** mMeshes;
public uint mNumMaterials;
public unsafe aiMaterial** mMaterials;
public uint mNumAnimations;
public unsafe aiAnimation** mAnimations;
public uint mNumTextures;
public unsafe aiTexture** mTextures;
public uint mNumLights;
public unsafe aiLight** mLights;
public uint mNumCameras;
public unsafe aiCamera** mCameras;
}
But many on this structs are managed ( aiNode, aiMesh, aiLight ) etc. So, I have this error:
Cannot take the address of, get the
size of, or declare a pointer to a
managed type ('Assimp.aiNode')
Any ideas on how to solve this issue?

This could get very complicated depending on what you are trying to do. However, as you are working it could help you do declare each of the pointers in the C# code like this. It uses the very useful IntPtr, which was described on here earlier today. :)
Please note that this will not magically get your code to work. I'd need to see a lot more of what's going on before I could give you input on that.
public struct aiScene
{
public uint Flags;
public IntPtr RootNode;
...
}

The main problem you're having is that you've defined managed objects with the same names as the unmanaged types. The pointer types, like "aiNode", should be defined as structs, not classes. You can define differently named managed wrapper classes which provide managed access to the underlying unsafe struct data. For example:
public struct aiNode {}
public struct aiScene
{
public uint mFlags;
public unsafe aiNode* mRootNode;
// ...
}
At a high level, it looks like you're trying to use AssImp from C#. That can be done today using assimp-net. However, in case someone runs into a marshalling situation like this and wants a generic answer..
I strongly recommend against using IntPtr for anything, because they are basically an untyped void* with no type checking. Unsafe-struct-pointers offer a safer disambiguation of unmanaged pointer types. SafeHandle is another option with better safety and handling of some race-conditions. See my article on the topic.
If you literally want to copy the data from unmanaged to managed land, then for each type (aiNode, aiMesh, etc) you need to define both an unsafe-struct (to match the unmanaged layout), and a managed class. Then, write (or generate) unsafe code to copy the unmanaged tree to managed objects. Be sure to be careful of any unmanaged objects with more than one reference.
Sometimes a better alternative is to write a wrapper which provides access to unmanaged data "in-place". If you are doing it "safely", then the managed wrappers should be thin-objects which manage lifetime for the unmanaged objects and have properties to access their data. Alternatively, you can do it unsafely by merely defining the unsafe structs and using them in an unsafe context.