Develop Reference

PInvoke marshalling const char * fails with UnmanagedType.LPStr

I'm failing to understand why the .NET Core marshalling is failing to work with the one marshakking approach but not another. In the example below Marshal.PtrToStringAnsi can successfully marshal a string from the native dll but the same call with the [return: MarshalAs(UnmanagedType.LPStr)] attribute cannot? Or is the marshalling from the IntPtr just getting lucky and could also access invalid memory depending on the OS memory management?
The C library code is (compiled as C not C++):
static const char str[] = "Hello World";
__declspec(dllexport) const char* getstr() {
return str;
}
C# test program:
using System;
using System.Runtime.InteropServices;
namespace CLibraryPinvoke
{
class Program
{
public const string LIB_NAME = "CLibrary";
[DllImport(LIB_NAME, EntryPoint = "getstr")]
public static extern IntPtr getstrgood();
[DllImport(LIB_NAME, EntryPoint ="getstr")]
[return: MarshalAs(UnmanagedType.LPStr)]
public static extern string getstrbad();
static void Main(string[] args)
{
Console.WriteLine($"getstr={Marshal.PtrToStringAnsi(getstrgood())}.");
// Crash.
Console.WriteLine($"getstr={getstrbad()}.");
}
}
}
Execution result:
getstr=Hello World.
CLibraryPinvoke\bin\Debug\netcoreapp3.1\CLibraryPinvoke.exe (process 31848) exited with code -1073740940.

PtrToStringAnsi copies the string from the buffer pointed to by the return value.
UnmanagedType.LPStr copies the string from the buffer pointed to by the return value, and releases it with CoTaskMemFree. Releasing a static const char[] with CoTaskMemFree crashes the program.
This is because the semantic of the return value is that the memory that holds the result was allocated on the called side.
As documented, if the memory is allocated on the called side, but should not be released with CoTaskMemFree, you must marshal it as IntPtr and release it yourself using the appropriate method.
The appropriate method for memory backed up by a static const char[] is to leave it alone.
This requires the user of your function to have knowledge of implementation details of your function, which is not a good thing. Of course, you can document the function as returning memory that should never be released, but then it would probably not be a very useful function, and if you were to change its behaviour in the future, you would be stuck with returning one of the possible static const char[]s.
You can avoid all that by allocating the memory each time and letting the .NET runtime to manage the rest, or by making the function accept a char* argument to copy the response into.

Converting std::vector to array an then p/Invoking it causes a Access Violation Exception in mscorlib.dll during marshalling

In C++ I have the following struct from 3rd-party code:
typedef struct NodeInfoTag
{
long lResult;
int bComplete;
char *pszNodeAddr;
char *pszParentAddr;
RTS_WCHAR *pwszNodeName;
RTS_WCHAR *pwszDeviceName;
RTS_WCHAR *pwszVendorName;
unsigned long ulTargetType;
unsigned long ulTargetId;
unsigned long ulTargetVersion;
unsigned short wMaxChannels;
}NodeInfotyp;
And the definition to RTS_WCHAR:
# ifndef RTS_WCHAR_DEFINED
# define RTS_WCHAR_DEFINED
typedef wchar_t RTS_WCHAR; /* wide character value */
# endif
(So it's basically a wchar_t)
Then I have my own class called CScanNetworkCallback, which extends the CPLCHandlerCallback class, a class from the same vendor:
.h file:
class CScanNetworkCallback : public CPLCHandlerCallback
{
public:
bool bScanComplete;
NodeInfotyp* pNodeInfo;
NodeInfotyp* pNodeInfoList;
std::vector<NodeInfotyp> vList;
CScanNetworkCallback();
virtual ~CScanNetworkCallback(void);
virtual long Notify(CPLCHandler *pPlcHandler, CallbackAddInfoTag CallbackAdditionalInfo);
};
The implementation follows their own guidelines with some of my own stuff thrown in:
CScanNetworkCallback::CScanNetworkCallback(void) : CPLCHandlerCallback()
{
bScanComplete = false;
}
CScanNetworkCallback::~CScanNetworkCallback()
{
delete pNodeInfo;
delete pNodeInfoList;
}
long CScanNetworkCallback::Notify(CPLCHandler *pPlcHandler, CallbackAddInfoTag CallbackAdditionalInfo)
{
if (pPlcHandler != NULL)
{
if (CallbackAdditionalInfo.ulType == PLCH_SCAN_NETWORK_CALLBACK)
{
pNodeInfo = CallbackAdditionalInfo.AddInf.pNodeInfo;
if (pNodeInfo->lResult == RESULT_OK)
{
vList.push_back(*pNodeInfo);
bScanComplete = false;
}
else
{
pNodeInfoList = &vList[0]; //New pointer points to the vector elements, which will be used as an array later on
// I have also tried copying it, to the same result:
//std::copy(vList.begin(), vList.end(), pNodeInfoList);
bScanComplete = true;
}
}
}
return RESULT_OK;
}
So basically, the Notify method in the class above is called every time a "node" is found in the network, assigning the node's information to pNodeInfo (please disregard what a node is, it isn't relevant ATM). Since it is called to every node in the network during the scanning process and I must send this information to C++, I couldn't find any other way to do so other than using a std::vector to store every callback info for latter use, as I don't know how many nodes there will be at compile time. The else part is called after all nodes have been found. In order to make sense out of the C# code, I must describe the implementation of some other C++ methods that are p/Invoked:
PROASADLL __declspec(dllexport) void scanNetwork(){
pScanHandler->ScanNetwork(NULL, &scanNetworkCallback);
}
The object scanNetworkCallback is static. pScanHandler is a pointer to another class from the 3rd party vendor and its ScanNetwork method runs on a separate thread. Internally (and I only know that due to this API Guidelines, I don't have its source code), it calls the Notify method whenever a node is found in the network, or something to that effect
And finally:
PROASADLL __declspec(dllexport) NodeInfotyp* getScanResult(int* piSize) {
*piSize = scanNetworkCallback.vList.size();
return scanNetworkCallback.pNodeInfoList;
}
That returns the pointer that points to all nodes' information and the amount in as an out parameter. Now let's take a look at the C# code:
public static List<NodeInfoTag> AsaScanNetworkAsync()
{
Console.WriteLine("SCANNING NETWORK");
scanNetwork(); // C++ Method
while (!isScanComplete()) // Holds the C# thread until the scan is complete
Thread.Sleep(50);
int size = 0;
IntPtr pointer = getScanResult(out size); // works fine, I get some IntPtr and the correct size
List<NodeInfoTag> list = Marshaller.MarshalPointerToList<NodeInfoTag>(pointer, size); // PROBLEM!!!
// Continue doing stuff
}
This is the class NodeInfoTag, to match the C++ NodeInfotyp struct:
[StructLayout(LayoutKind.Sequential)]
public class NodeInfoTag
{
public int Result;
public int Complete;
[MarshalAs(UnmanagedType.LPStr)] //char*
public string NodeAddress;
[MarshalAs(UnmanagedType.LPStr)] //char*
public string ParentAddress;
[MarshalAs(UnmanagedType.LPWStr)] //wchar_t
public string VendorName;
public uint TargetType;
public uint TargetId;
public uint TargetVersion;
public short MaxChannels;
}
And this is where I get my Memory Access Violation:
internal class Marshaller
{
public static List<T> MarshalPointerToList<T>(IntPtr pointer, int size)
{
if (size == 0)
return null;
List<T> list = new List<T>();
var symbolSize = Marshal.SizeOf(typeof(T));
for (int i = 0; i < size; i++)
{
var current = (T)Marshal.PtrToStructure(pointer, typeof(T));
list.Add(current);
pointer = new IntPtr(pointer.ToInt32() + symbolSize);
}
return list;
}
}
The error occurs specifically when marshaling should take place, at the line var current = (T)Marshal.PtrToStructure(pointer, typeof(T));. This C# code used to work just fine, but the C++ part was terrible, convoluted and error-prone, so I decided to make things more simple but I can't figure out for the life of me why I'm getting this Exception as I'm making sure that all C++ resources are available for C#, since for testing purposes, I don't delete anything in C++ and I'm only using variables with global scope within the class, which is allocated to static memory. So, what did I miss?
Edit: I removed pNodeInfoList = &vList[0]; and rewrote getScanResult as follows:
static NodeInfotyp pNodeInfoList;
//(...)
PROASADLL __declspec(dllexport) NodeInfotyp* getScanResult(int* piSize) {
*piSize = scanNetworkCallback.vList.size();
std::move(scanNetworkCallback.vList.begin(),
scanNetworkCallback.vList.end(), &pNodeInfoList);
return &pNodeInfoList;
}
No dice. I don't use new or malloc in any of the variables involved, and even changed pNodeInfoList (the array) from a class member to a global variable. Also, I'm using move, as I've been told, could be used to solve ownership problems. Any other tips?

Ownership is not part of the naive C++ type system, so you will not get an error when you delete a pointer you do not own or transfer ownership away without giving it up.
However, semantically certain values and pointers and data blocks are owned by certain types or values.
In this case the vector owns its block of memory. There is no way to ask it or make it give up ownership.
Calling .data() onky provides you a pointer, it does not give that pointer semantic ownership.
You store the return value of .data() in a member variable. You later call delete on that member variable. This indicates to me that member variable is supposed to own its data. So you double delete (as both the vector and the pointer think they own the data pointed to), and your compiler crashes the program for you.
You need to rewite your code taking into account liefetime and ownership of every block of memory you are working with. One approach is to never ever call new, malloc or delete or free directly, and always use memory managing types like vector and unique ptr. Avoid persisting raw pointers, as their ownership semantics are not clear from the type.

Marshalling a char** in C#

I am interfacing with code that takes a char** (that is, a pointer to a string):
int DoSomething(Whatever* handle, char** error);
Basically, it takes a handle to its state, and if something goes wrong, it returns an error code and optionally an error message (the memory is allocated externally and freed with a second function. That part I've figued out :) ).
I, however, am unsure how to handle in in C#. What I have currently:
[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
private static unsafe extern int DoSomething(IntPtr handle, byte** error);
public static unsafe int DoSomething(IntPtr handle, out string error) {
byte* buff;
int ret = DoSomething(handle, &buff);
if(buff != 0) {
// ???
} else {
error = "";
}
return ret;
}
I've poked around, but I can't figure out how to turn that into a byte[], suitable for feeding to UTF8Encoding.UTF8.GetString()
Am I on the right track?
EDIT: To make more explicit, the library function allocates memory, which must be freed by calling another library function. If a solution does not leave me with a pointer I can free, the solution is unacceptable.
Bonus question: As implied above, this library uses UTF-8 for its strings. Do I need to do anything special in my P/Invokes, or just use string for normal const char* parameters?

You should just be able to use a ref string and have the runtime default marshaller take care of this conversion for you. You can hint the char width on the parameter with [MarshalAs(UnmanagedType.LPStr)] to make sure that you are using 8-bit characters.
Since you have a special deallocation method to call, you'll need to keep the pointer, like you've already shown in your question's example.
Here's how I'd write it:
[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
private static unsafe extern int DoSomething(
MySafeHandle handle, void** error); // byte** should work, too, I'm just lazy
Then you can get a string:
var errorMsg = Marshal.PtrToStringAnsi(new IntPtr(*error));
And cleanup:
[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
private static extern int FreeMyMemory(IntPtr h);
// ...
FreeMyMemory(new IntPtr(error));
And now we have the marshalled error, so just return it.
return errorMsg;
Also note the MySafeHandle type, which would inherit from System.Runtime.InteropServices.SafeHandle. While not strictly needed (you can use IntPtr), it gives you a better handle management when interoping with native code. Read about it here: http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.safehandle.aspx.

For reference, here is code that compiles (but, not tested yet, working on that next tested, works 100%) that does what I need. If anyone can do better, that's what I'm after :D
public static unsafe int DoSomething(IntPtr handle, out string error) {
byte* buff;
int ret = DoSomething(handle, &buff);
if(buff != null) {
int i = 0;
//count the number of bytes in the error message
while (buff[++i] != 0) ;
//allocate a managed array to store the data
byte[] tmp = new byte[i];
//(Marshal only works with IntPtrs)
IntPtr errPtr = new IntPtr(buff);
//copy the unmanaged array over
Marshal.Copy(buff, tmp, 0, i);
//get the string from the managed array
error = UTF8Encoding.UTF8.GetString(buff);
//free the unmanaged array
//omitted, since it's not important
//take a shot of whiskey
} else {
error = "";
}
return ret;
}
Edit: fixed the logic in the while loop, it had an off by one error.

How can I marshall a vector<int> from a C++ dll to a C# application?

I have a C++ function that produces a list of rectangles that are interesting. I want to be able to get that list out of the C++ library and back into the C# application that is calling it.
So far, I'm encoding the rectangles like so:
struct ImagePatch{
int xmin, xmax, ymin, ymax;
}
and then encoding some vectors:
void MyFunc(..., std::vector<int>& rectanglePoints){
std::vector<ImagePatch> patches; //this is filled with rectangles
for(i = 0; i < patches.size(); i++){
rectanglePoints.push_back(patches[i].xmin);
rectanglePoints.push_back(patches[i].xmax);
rectanglePoints.push_back(patches[i].ymin);
rectanglePoints.push_back(patches[i].ymax);
}
}
The header for interacting with C# looks like (and works for a bunch of other functions):
extern "C" {
__declspec(dllexport) void __cdecl MyFunc(..., std::vector<int>& rectanglePoints);
}
Are there some keywords or other things I can do to get that set of rectangles out? I found this article for marshalling objects in C#, but it seems way too complicated and way too underexplained. Is a vector of integers the right way to do this, or is there some other trick or approach?

The STL is a C++ specific library, so you cant directly get it across as one object to C#.
The one thing that is guaranteed about std::vector is that &v[0] points to the first element and all the elements lie linearly in memory (in other words, its just like a C array in terms of memory layout)
So marshal as array of int... which shouldn't be hard - There are lot of examples on the web.
Added
Assuming you only pass the data from C++ to C# :
C# cannot handle a C++ vector object, so do not try passing it by reference : Instead your C++ code must return a pointer to an array of ints...
If you are not going to be using this function from multiple threads, you can use static storage :
int *getRects(bool bClear)
{
static vector<int> v; // This variable persists across invocations
if(bClear)
{
v.swap(vector<int>());
}
else
{
v.clear();
// Fill v with data as you wish
}
return v.size() ? &v[0] : NULL;
}
call getRects(true) if the returned data is significant in size, so you release the memory in v.
For simplicity, instead of passing out the size of the vector data too, just put a sentinel value at the end (like say -1) so the C# code can detect where the data ends.

Yes. You can. Actually, not just std::vector, std::string, std::wstring, any standard C++ class or your own classes can be marshaled or instantiated and called from C#/.NET.
Wrapping a std::vector<any_type> in C# is indeed possible with just regular P/Invoke Interop, it is complicated though. even a std::map of any type can be done in C#/.NET.
public class SampleClass : IDisposable
{
[DllImport("YourDll.dll", EntryPoint="ConstructorOfYourClass", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void SampleClassConstructor(IntPtr thisObject);
[DllImport("YourDll.dll", EntryPoint="DestructorOfYourClass", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void SampleClassDestructor(IntPtr thisObject);
[DllImport("YourDll.dll", EntryPoint="DoSomething", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void DoSomething(IntPtr thisObject);
[DllImport("YourDll.dll", EntryPoint="DoSomethingElse", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void DoSomething(IntPtr thisObject, int x);
IntPtr ptr;
public SampleClass(int sizeOfYourCppClass)
{
this.ptr = Marshal.AllocHGlobal(sizeOfYourCppClass);
SampleClassConstructor(this.ptr);
}
public void DoSomething()
{
DoSomething(this.ptr);
}
public void DoSomethingElse(int x)
{
DoSomethingElse(this.ptr, x);
}
public void Dispose()
{
if (this.ptr != IntPtr.Zero)
{
// The following 2 calls equals to "delete object" in C++
// Calling the destructor of the C++ class will free the memory allocated by the native c++ class.
SampleClassDestructor(this.ptr);
// Free the memory allocated from .NET.
Marshal.FreeHGlobal(this.ptr);
this.ptr = IntPtr.Zero;
}
}
}
Please see the below link,
C#/.NET PInvoke Interop SDK
(I am the author of the SDK tool)
Once you have the C# wrapper class for your C++ class ready, it is easy to implement ICustomMarshaler so that you can marshal the C++ object from .NET.
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.icustommarshaler.aspx

I'm pretty sure you can't do this. You have to be able to translate the C++ code directly to a C# class, so you would at least have to replicate the internals of the vector class to marshall it correctly. I'm also pretty sure you won't be able to move references across the boundary, you'll have to use IntPtr (raw pointers). The approach that i know works is to marshall a raw array of the structs.

Invalid Variant crash

I have a situation where I've wrapped a Native C++ DLL with C++/CLI for eventual use in C#.
There are a few callback functions that are causing some issues at run time. Particularly, I get the following exception:
An unhandled exception of type
'System.Runtime.InteropServices.InvalidOleVariantTypeException'
occurred in ToadWrapTest.dll
Additional information: Specified OLE
variant is invalid.
On this line of code (C++/CLI):
public delegate int ManagedCallbackFunction (Object^ inst, const Object^ data);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
ManagedCallbackFunction^ m_callbackFn;
int intermidiaryCallback(void * pInstance, const void * pData)
{
void* temp = (void*)pData;
System::IntPtr ip1 = IntPtr(pInstance);
System::IntPtr ip2 = IntPtr(temp);
Object^ oInst = Marshal::GetObjectForNativeVariant(ip1);
Object^ oData = Marshal::GetObjectForNativeVariant(ip2);
//invoke the callback to c#
//return m_callbackFn::Invoke(oInst, oData);
return 0;
};
The reason I've made this "intermediary callback" was an attempt to circumvent the Invalid variant exception being thrown when I tried to directly map the delegate from C# to the native C++ code. As an attempted work-around, I declare a delegate on the C# side and pass that funcptr to the C++/CLI wrapper. I then pass the intermediary funcptr to the native C++ and just daisy chain the calls together.
What I know is that it all works in native C++ world. The problem is mapping the void* to the managed world. The following code shows the native C++ version of the callback:
int (*CallbackFunction) (void *inst, const void *data);
If anyone can help here, I'd really appreciate it.

Are pInstance and pData really VARIANT? If they are, I would expect your callback function to be more strongly typed:
int (*CallbackFunction)(VARIANT *inst, VARIANT *data);
If that's the case, in your code you should be able to look at the actual VARIANT to hand check it. If you are not really getting VARIANTs (ie, you are really just getting void * pointers), you shouldn't try to turn them into C# objects since there is no inherent meaning to them. They should get passed through as IntPtr. If you know that they should have some other type of inherent meaning, you need to marshal them as appropriate types.

Big Thanks to plinth on this one! I am posting the final solution below to anyone else who has to deal with 3rd party fun like this one! Please feel free to critique, as I am not done optimizing the code. This may still be to roundabout a solution.
First, the callback functions became:
public delegate int ManagedCallbackFunction (IntPtr oInst, IntPtr oData);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
ManagedCallbackFunction^ m_callbackFn;
Big props on this one. It just plain won't work if you try to cast from void* directly to Object^. Using the IntPtr and my intermediary callback:
int intermidiaryCallback(void * pInstance, const void * pData)
{
void* temp = (void*)pData;
return m_callbackFn->Invoke(IntPtr(pInstance), IntPtr(temp));
};
We finally get a working model on the C# side with some massaging of the objects:
public static int hReceiveTestMessage(IntPtr pInstance, IntPtr pData)
{
// provide object context for static member function
helloworld2 hw = (helloworld2)GCHandle.FromIntPtr(pInstance).Target;
if (hw == null || pData == null)
{
Console.WriteLine("hReceiveTestMessage received NULL data or instance pointer\n");
return 0;
}
// populate message with received data
IntPtr ip2 = GCHandle.ToIntPtr(GCHandle.Alloc(new DataPacketWrap(pData)));
DataPacketWrap dpw = (DataPacketWrap)GCHandle.FromIntPtr(ip2).Target;
uint retval = hw.m_testData.load_dataSets(ref dpw);
// display message contents
hw.displayTestData();
return 1;
}
I mention "massaging" the objects because the delegate is not specific to this callback function and I don't know what object pData will be until run time(from the delegates POV). Because of this issue, I have to do some extra work with the pData object. I basically had to overload the constructor in my wrapper to accept an IntPtr. Code is provided for full "clarity":
DataPacketWrap (IntPtr dp)
{
DataPacket* pdp = (DataPacket*)(dp.ToPointer());
m_NativeDataPacket = pdp;
};