Develop Reference

Converting std::vector to array an then p/Invoking it causes a Access Violation Exception in mscorlib.dll during marshalling

In C++ I have the following struct from 3rd-party code:
typedef struct NodeInfoTag
{
long lResult;
int bComplete;
char *pszNodeAddr;
char *pszParentAddr;
RTS_WCHAR *pwszNodeName;
RTS_WCHAR *pwszDeviceName;
RTS_WCHAR *pwszVendorName;
unsigned long ulTargetType;
unsigned long ulTargetId;
unsigned long ulTargetVersion;
unsigned short wMaxChannels;
}NodeInfotyp;
And the definition to RTS_WCHAR:
# ifndef RTS_WCHAR_DEFINED
# define RTS_WCHAR_DEFINED
typedef wchar_t RTS_WCHAR; /* wide character value */
# endif
(So it's basically a wchar_t)
Then I have my own class called CScanNetworkCallback, which extends the CPLCHandlerCallback class, a class from the same vendor:
.h file:
class CScanNetworkCallback : public CPLCHandlerCallback
{
public:
bool bScanComplete;
NodeInfotyp* pNodeInfo;
NodeInfotyp* pNodeInfoList;
std::vector<NodeInfotyp> vList;
CScanNetworkCallback();
virtual ~CScanNetworkCallback(void);
virtual long Notify(CPLCHandler *pPlcHandler, CallbackAddInfoTag CallbackAdditionalInfo);
};
The implementation follows their own guidelines with some of my own stuff thrown in:
CScanNetworkCallback::CScanNetworkCallback(void) : CPLCHandlerCallback()
{
bScanComplete = false;
}
CScanNetworkCallback::~CScanNetworkCallback()
{
delete pNodeInfo;
delete pNodeInfoList;
}
long CScanNetworkCallback::Notify(CPLCHandler *pPlcHandler, CallbackAddInfoTag CallbackAdditionalInfo)
{
if (pPlcHandler != NULL)
{
if (CallbackAdditionalInfo.ulType == PLCH_SCAN_NETWORK_CALLBACK)
{
pNodeInfo = CallbackAdditionalInfo.AddInf.pNodeInfo;
if (pNodeInfo->lResult == RESULT_OK)
{
vList.push_back(*pNodeInfo);
bScanComplete = false;
}
else
{
pNodeInfoList = &vList[0]; //New pointer points to the vector elements, which will be used as an array later on
// I have also tried copying it, to the same result:
//std::copy(vList.begin(), vList.end(), pNodeInfoList);
bScanComplete = true;
}
}
}
return RESULT_OK;
}
So basically, the Notify method in the class above is called every time a "node" is found in the network, assigning the node's information to pNodeInfo (please disregard what a node is, it isn't relevant ATM). Since it is called to every node in the network during the scanning process and I must send this information to C++, I couldn't find any other way to do so other than using a std::vector to store every callback info for latter use, as I don't know how many nodes there will be at compile time. The else part is called after all nodes have been found. In order to make sense out of the C# code, I must describe the implementation of some other C++ methods that are p/Invoked:
PROASADLL __declspec(dllexport) void scanNetwork(){
pScanHandler->ScanNetwork(NULL, &scanNetworkCallback);
}
The object scanNetworkCallback is static. pScanHandler is a pointer to another class from the 3rd party vendor and its ScanNetwork method runs on a separate thread. Internally (and I only know that due to this API Guidelines, I don't have its source code), it calls the Notify method whenever a node is found in the network, or something to that effect
And finally:
PROASADLL __declspec(dllexport) NodeInfotyp* getScanResult(int* piSize) {
*piSize = scanNetworkCallback.vList.size();
return scanNetworkCallback.pNodeInfoList;
}
That returns the pointer that points to all nodes' information and the amount in as an out parameter. Now let's take a look at the C# code:
public static List<NodeInfoTag> AsaScanNetworkAsync()
{
Console.WriteLine("SCANNING NETWORK");
scanNetwork(); // C++ Method
while (!isScanComplete()) // Holds the C# thread until the scan is complete
Thread.Sleep(50);
int size = 0;
IntPtr pointer = getScanResult(out size); // works fine, I get some IntPtr and the correct size
List<NodeInfoTag> list = Marshaller.MarshalPointerToList<NodeInfoTag>(pointer, size); // PROBLEM!!!
// Continue doing stuff
}
This is the class NodeInfoTag, to match the C++ NodeInfotyp struct:
[StructLayout(LayoutKind.Sequential)]
public class NodeInfoTag
{
public int Result;
public int Complete;
[MarshalAs(UnmanagedType.LPStr)] //char*
public string NodeAddress;
[MarshalAs(UnmanagedType.LPStr)] //char*
public string ParentAddress;
[MarshalAs(UnmanagedType.LPWStr)] //wchar_t
public string VendorName;
public uint TargetType;
public uint TargetId;
public uint TargetVersion;
public short MaxChannels;
}
And this is where I get my Memory Access Violation:
internal class Marshaller
{
public static List<T> MarshalPointerToList<T>(IntPtr pointer, int size)
{
if (size == 0)
return null;
List<T> list = new List<T>();
var symbolSize = Marshal.SizeOf(typeof(T));
for (int i = 0; i < size; i++)
{
var current = (T)Marshal.PtrToStructure(pointer, typeof(T));
list.Add(current);
pointer = new IntPtr(pointer.ToInt32() + symbolSize);
}
return list;
}
}
The error occurs specifically when marshaling should take place, at the line var current = (T)Marshal.PtrToStructure(pointer, typeof(T));. This C# code used to work just fine, but the C++ part was terrible, convoluted and error-prone, so I decided to make things more simple but I can't figure out for the life of me why I'm getting this Exception as I'm making sure that all C++ resources are available for C#, since for testing purposes, I don't delete anything in C++ and I'm only using variables with global scope within the class, which is allocated to static memory. So, what did I miss?
Edit: I removed pNodeInfoList = &vList[0]; and rewrote getScanResult as follows:
static NodeInfotyp pNodeInfoList;
//(...)
PROASADLL __declspec(dllexport) NodeInfotyp* getScanResult(int* piSize) {
*piSize = scanNetworkCallback.vList.size();
std::move(scanNetworkCallback.vList.begin(),
scanNetworkCallback.vList.end(), &pNodeInfoList);
return &pNodeInfoList;
}
No dice. I don't use new or malloc in any of the variables involved, and even changed pNodeInfoList (the array) from a class member to a global variable. Also, I'm using move, as I've been told, could be used to solve ownership problems. Any other tips?

Ownership is not part of the naive C++ type system, so you will not get an error when you delete a pointer you do not own or transfer ownership away without giving it up.
However, semantically certain values and pointers and data blocks are owned by certain types or values.
In this case the vector owns its block of memory. There is no way to ask it or make it give up ownership.
Calling .data() onky provides you a pointer, it does not give that pointer semantic ownership.
You store the return value of .data() in a member variable. You later call delete on that member variable. This indicates to me that member variable is supposed to own its data. So you double delete (as both the vector and the pointer think they own the data pointed to), and your compiler crashes the program for you.
You need to rewite your code taking into account liefetime and ownership of every block of memory you are working with. One approach is to never ever call new, malloc or delete or free directly, and always use memory managing types like vector and unique ptr. Avoid persisting raw pointers, as their ownership semantics are not clear from the type.

Exception when calling C++ function from managed code

Let's imagine C++ function in DLL:
typedef struct
{
int number;
void *list;
} clist_t;
extern "C" DLL_API clist_t GetItems(int a);
DLL_API clist_t GetItems(int a)
{
static clist_t list;
list.number = a;
list.list = sth();
return list;
}
And the C# code which calls this function:
[StructLayout(LayoutKind.Sequential)]
public struct CList
{
public int number;
public IntPtr ptr;
};
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate CList GetItemsDelegate(int a);
int func = Win32APIWrapper.GetProcAddress(m_dllHndl, "GetItems");
private GetItemsDelegate GeItemsFunc =
(GetItmsDelegate)Marshal.GetDelegateForFunctionPointer((IntPtr)func, typeof(GetItemsDelegate));
CList list = GetItemsFunc(12); // <--- here we got exception
The last function throws exception: "Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
I noticed that removing second field in CList function causes exception to disappear and returned struct has correct value of number field.
So why the original code does not work? The alignment of structures seems to be correct. Also calling convention is properly set to cdecl.
Any ideas? Both types in struct are blittable, so there is no need to even marshal them (C# and dll are compiled to 32bit).
EDIT:
Why returning smaller struct does not throw exception? I noticed that struct smaller then 4 bytes are returned ok. On the other hand the same function but without parameters allows returning the original struct without exception.

How can I marshall a vector<int> from a C++ dll to a C# application?

I have a C++ function that produces a list of rectangles that are interesting. I want to be able to get that list out of the C++ library and back into the C# application that is calling it.
So far, I'm encoding the rectangles like so:
struct ImagePatch{
int xmin, xmax, ymin, ymax;
}
and then encoding some vectors:
void MyFunc(..., std::vector<int>& rectanglePoints){
std::vector<ImagePatch> patches; //this is filled with rectangles
for(i = 0; i < patches.size(); i++){
rectanglePoints.push_back(patches[i].xmin);
rectanglePoints.push_back(patches[i].xmax);
rectanglePoints.push_back(patches[i].ymin);
rectanglePoints.push_back(patches[i].ymax);
}
}
The header for interacting with C# looks like (and works for a bunch of other functions):
extern "C" {
__declspec(dllexport) void __cdecl MyFunc(..., std::vector<int>& rectanglePoints);
}
Are there some keywords or other things I can do to get that set of rectangles out? I found this article for marshalling objects in C#, but it seems way too complicated and way too underexplained. Is a vector of integers the right way to do this, or is there some other trick or approach?

The STL is a C++ specific library, so you cant directly get it across as one object to C#.
The one thing that is guaranteed about std::vector is that &v[0] points to the first element and all the elements lie linearly in memory (in other words, its just like a C array in terms of memory layout)
So marshal as array of int... which shouldn't be hard - There are lot of examples on the web.
Added
Assuming you only pass the data from C++ to C# :
C# cannot handle a C++ vector object, so do not try passing it by reference : Instead your C++ code must return a pointer to an array of ints...
If you are not going to be using this function from multiple threads, you can use static storage :
int *getRects(bool bClear)
{
static vector<int> v; // This variable persists across invocations
if(bClear)
{
v.swap(vector<int>());
}
else
{
v.clear();
// Fill v with data as you wish
}
return v.size() ? &v[0] : NULL;
}
call getRects(true) if the returned data is significant in size, so you release the memory in v.
For simplicity, instead of passing out the size of the vector data too, just put a sentinel value at the end (like say -1) so the C# code can detect where the data ends.

Yes. You can. Actually, not just std::vector, std::string, std::wstring, any standard C++ class or your own classes can be marshaled or instantiated and called from C#/.NET.
Wrapping a std::vector<any_type> in C# is indeed possible with just regular P/Invoke Interop, it is complicated though. even a std::map of any type can be done in C#/.NET.
public class SampleClass : IDisposable
{
[DllImport("YourDll.dll", EntryPoint="ConstructorOfYourClass", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void SampleClassConstructor(IntPtr thisObject);
[DllImport("YourDll.dll", EntryPoint="DestructorOfYourClass", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void SampleClassDestructor(IntPtr thisObject);
[DllImport("YourDll.dll", EntryPoint="DoSomething", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void DoSomething(IntPtr thisObject);
[DllImport("YourDll.dll", EntryPoint="DoSomethingElse", CharSet=CharSet.Ansi, CallingConvention=CallingConvention.ThisCall)]
public extern static void DoSomething(IntPtr thisObject, int x);
IntPtr ptr;
public SampleClass(int sizeOfYourCppClass)
{
this.ptr = Marshal.AllocHGlobal(sizeOfYourCppClass);
SampleClassConstructor(this.ptr);
}
public void DoSomething()
{
DoSomething(this.ptr);
}
public void DoSomethingElse(int x)
{
DoSomethingElse(this.ptr, x);
}
public void Dispose()
{
if (this.ptr != IntPtr.Zero)
{
// The following 2 calls equals to "delete object" in C++
// Calling the destructor of the C++ class will free the memory allocated by the native c++ class.
SampleClassDestructor(this.ptr);
// Free the memory allocated from .NET.
Marshal.FreeHGlobal(this.ptr);
this.ptr = IntPtr.Zero;
}
}
}
Please see the below link,
C#/.NET PInvoke Interop SDK
(I am the author of the SDK tool)
Once you have the C# wrapper class for your C++ class ready, it is easy to implement ICustomMarshaler so that you can marshal the C++ object from .NET.
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.icustommarshaler.aspx

I'm pretty sure you can't do this. You have to be able to translate the C++ code directly to a C# class, so you would at least have to replicate the internals of the vector class to marshall it correctly. I'm also pretty sure you won't be able to move references across the boundary, you'll have to use IntPtr (raw pointers). The approach that i know works is to marshall a raw array of the structs.

Pinning an updateble struct before passing to unmanaged code?

I using some old API and need to pass the a pointer of a struct to unmanaged code that runs asynchronous.
In other words, after i passing the struct pointer to the unmanaged code, the unmanaged code copies the pointer and returns immediately. The unmanaged code can access that struct in background, in another thread. I have no control over the unmanaged code that runs in another thread nor the thread itself.
The fixed { } statement can't be used for pinning because it not designed for async unmanaged pinning.
GCHandle can pin only references, so the struct must be boxed to use GCHandle. I tried it, and it works. The main problem with it, is that you can't update the struct from managed code. To update a struct, first of all we need to unbox it, then update, then box again, but... oops ... box again?!? this means the previous pointer in the memory still point to the old non-up-to-date struct, and the new struct have another pointer, and this means that i need to pass new pointer to the unmanaged code... inapplicable in my case.
How can i pin a struct in the memory without fixed { } statement, and so that i can update it from managed code without change it's pointer?
Thanks.
Edit:
Just thought... is there a way to pin the parent object that contains the struct, and then get the pointer of the struct rather than the container object?

Using pinned memory in this case is not a good idea, given that the memory for the struct needs to be valid for a long time. GCHandle.Alloc() will box the structure and store it on the heap. With it being pinned, it will be a long term burden to the garbage collector as it needs to constantly find a way around the rock in the road.
The simple solution is to allocate memory for the struct in unmanaged memory. Use Marshal.SizeOf() to get the size of the structure and Marshal.AllocCoTaskMem() to allocate the memory. That gets you the pointer you need to pass to the unmanaged code. Initialize the memory with Marshal.StructureToPtr(). And read updates to the structure written by the unmanaged code with PtrToStructure().
If you do this frequently, you'll be constantly copying the structure. That could be expensive, depending on the size of the structure. To avoid that, use an unsafe pointer to access the unmanaged memory directly. Some basic syntax:
using System;
using System.Runtime.InteropServices;
class Program {
unsafe static void Main(string[] args) {
int len = Marshal.SizeOf(typeof(Test));
IntPtr mem = Marshal.AllocCoTaskMem(len);
Test* ptr = (Test*)mem;
ptr->member1 = 42;
// call method
//..
int value = ptr->member1;
Marshal.FreeCoTaskMem(mem);
}
public struct Test {
public int member1;
}
}

Is unsafe code an option?
// allocate unmanaged memory
Foo* foo = (Foo*)Marshal.AllocHGlobal(sizeof(Foo));
// initialize struct
foo->bar = 0;
// invoke unmanaged function which remembers foo
UnsafeNativeMethods.Bar(foo);
Console.WriteLine(foo->bar);
// update struct
foo->bar = 10;
// invoke unmanaged function which uses remembered foo
UnsafeNativeMethods.Qux();
Console.WriteLine(foo->bar);
// free unmanaged memory
Marshal.FreeHGlobal((IntPtr)foo);
This compiles and doesn't throw an exception, but I don't have an unmanaged function at hand to test if it works.
From MSDN:
When AllocHGlobal calls LocalAlloc, it passes a LMEM_FIXED flag, which causes the allocated memory to be locked in place. Also, the allocated memory is not zero-filled.

Instead of pinning, you need to use Marshal.StructureToPtr and Marshal.PtrToStructure to marshal the struct into memory that's usable in native code.

Struct example:
[StructLayout(LayoutKind.Sequential)]
public struct OVERLAPPED_STRUCT
{
public IntPtr InternalLow;
public IntPtr InternalHigh;
public Int32 OffsetLow;
public Int32 OffsetHigh;
public IntPtr EventHandle;
}
How to pin it to the struct and use it:
OVERLAPPED_STRUCT over_lapped = new OVERLAPPED_STRUCT();
// edit struct in managed code
over_lapped.OffsetLow = 100;
IntPtr pinned_overlap_struct = Marshal.AllocHGlobal(Marshal.SizeOf(over_lapped));
Marshal.StructureToPtr(over_lapped, pinned_overlap_struct, true);
// Pass pinned_overlap_struct to your unmanaged code
// pinned_overlap_struct changes ...
// Get resulting new struct
OVERLAPPED_STRUCT nat_ov = (OVERLAPPED_STRUCT)Marshal.PtrToStructure(pinned_overlap_struct, typeof(OVERLAPPED_STRUCT));
// See what new value is
int offset_low = nat_ov.OffsetLow;
// Clean up
Marshal.FreeHGlobal(pinned_overlap_struct);

How about having the struct include an ActOnMe() interface and method something like:
delegate void ActByRef<T1,T2>(ref T1 p1, ref T2 p2);
interface IActOnMe<TT> {ActOnMe<T>(ActByRef<TT,T> proc, ref T param);}
struct SuperThing : IActOnMe<SuperThing>
{
int this;
int that;
...
void ActOnMe<T>(ActByRef<SuperThing,T>, ref T param)
{
proc(ref this, ref param);
}
}
Because the delegate takes a generic parameter by reference, it should be possible in most cases to avoid the overhead of creating closures by passing a delegate to a static method along with a reference to a struct to carry data to or from that method. Further, casting an already-boxed instance of SuperThing to IActOnMe<SuperThing> and calling ActOnMe<T> on it will expose the fields of that boxed instance for updating, as opposed to creating another copy of them as would occur with a typecast to the struct.

To answer your edit:
Just thought... is there a way to pin the parent object that contains the struct, and then get the pointer of the struct rather than the container object?
I think so. If anything, you should be able to with a managed array of structures (possibly an array of one).
Here is an example code:
[StructLayout(LayoutKind.Sequential)]
struct SomeStructure
{
public int first;
public int second;
public SomeStructure(int first, int second) { this.first=first; this.second=second; }
}
/// <summary>
/// For this question on Stack Overflow:
/// https://stackoverflow.com/questions/1850488/pinning-an-updateble-struct-before-passing-to-unmanaged-code
/// </summary>
private static void TestModifiableStructure()
{
SomeStructure[] objArray = new SomeStructure[1];
objArray[0] = new SomeStructure(10, 10);
GCHandle hPinned = GCHandle.Alloc(objArray, GCHandleType.Pinned);
//Modify the pinned structure, just to show we can
objArray[0].second = 42;
Console.WriteLine("Before unmanaged incrementing: {0}", objArray[0].second);
PlaceholderForUnmanagedFunction(hPinned.AddrOfPinnedObject());
Console.WriteLine("Before unmanaged incrementing: {0}", objArray[0].second);
//Cleanup
hPinned.Free();
}
//Simulates an unmanaged function that accesses ptr->second
private static void PlaceholderForUnmanagedFunction(IntPtr ptr)
{
int secondInteger = Marshal.ReadInt32(ptr, 4);
secondInteger++;
Marshal.WriteInt32(ptr, 4, secondInteger);
}
And its output:
Before unmanaged incrementing: 42
Before unmanaged incrementing: 43

Invalid Variant crash

I have a situation where I've wrapped a Native C++ DLL with C++/CLI for eventual use in C#.
There are a few callback functions that are causing some issues at run time. Particularly, I get the following exception:
An unhandled exception of type
'System.Runtime.InteropServices.InvalidOleVariantTypeException'
occurred in ToadWrapTest.dll
Additional information: Specified OLE
variant is invalid.
On this line of code (C++/CLI):
public delegate int ManagedCallbackFunction (Object^ inst, const Object^ data);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
ManagedCallbackFunction^ m_callbackFn;
int intermidiaryCallback(void * pInstance, const void * pData)
{
void* temp = (void*)pData;
System::IntPtr ip1 = IntPtr(pInstance);
System::IntPtr ip2 = IntPtr(temp);
Object^ oInst = Marshal::GetObjectForNativeVariant(ip1);
Object^ oData = Marshal::GetObjectForNativeVariant(ip2);
//invoke the callback to c#
//return m_callbackFn::Invoke(oInst, oData);
return 0;
};
The reason I've made this "intermediary callback" was an attempt to circumvent the Invalid variant exception being thrown when I tried to directly map the delegate from C# to the native C++ code. As an attempted work-around, I declare a delegate on the C# side and pass that funcptr to the C++/CLI wrapper. I then pass the intermediary funcptr to the native C++ and just daisy chain the calls together.
What I know is that it all works in native C++ world. The problem is mapping the void* to the managed world. The following code shows the native C++ version of the callback:
int (*CallbackFunction) (void *inst, const void *data);
If anyone can help here, I'd really appreciate it.

Are pInstance and pData really VARIANT? If they are, I would expect your callback function to be more strongly typed:
int (*CallbackFunction)(VARIANT *inst, VARIANT *data);
If that's the case, in your code you should be able to look at the actual VARIANT to hand check it. If you are not really getting VARIANTs (ie, you are really just getting void * pointers), you shouldn't try to turn them into C# objects since there is no inherent meaning to them. They should get passed through as IntPtr. If you know that they should have some other type of inherent meaning, you need to marshal them as appropriate types.

Big Thanks to plinth on this one! I am posting the final solution below to anyone else who has to deal with 3rd party fun like this one! Please feel free to critique, as I am not done optimizing the code. This may still be to roundabout a solution.
First, the callback functions became:
public delegate int ManagedCallbackFunction (IntPtr oInst, IntPtr oData);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
ManagedCallbackFunction^ m_callbackFn;
Big props on this one. It just plain won't work if you try to cast from void* directly to Object^. Using the IntPtr and my intermediary callback:
int intermidiaryCallback(void * pInstance, const void * pData)
{
void* temp = (void*)pData;
return m_callbackFn->Invoke(IntPtr(pInstance), IntPtr(temp));
};
We finally get a working model on the C# side with some massaging of the objects:
public static int hReceiveTestMessage(IntPtr pInstance, IntPtr pData)
{
// provide object context for static member function
helloworld2 hw = (helloworld2)GCHandle.FromIntPtr(pInstance).Target;
if (hw == null || pData == null)
{
Console.WriteLine("hReceiveTestMessage received NULL data or instance pointer\n");
return 0;
}
// populate message with received data
IntPtr ip2 = GCHandle.ToIntPtr(GCHandle.Alloc(new DataPacketWrap(pData)));
DataPacketWrap dpw = (DataPacketWrap)GCHandle.FromIntPtr(ip2).Target;
uint retval = hw.m_testData.load_dataSets(ref dpw);
// display message contents
hw.displayTestData();
return 1;
}
I mention "massaging" the objects because the delegate is not specific to this callback function and I don't know what object pData will be until run time(from the delegates POV). Because of this issue, I have to do some extra work with the pData object. I basically had to overload the constructor in my wrapper to accept an IntPtr. Code is provided for full "clarity":
DataPacketWrap (IntPtr dp)
{
DataPacket* pdp = (DataPacket*)(dp.ToPointer());
m_NativeDataPacket = pdp;
};