Invalid Variant crash - c#

I have a situation where I've wrapped a Native C++ DLL with C++/CLI for eventual use in C#.
There are a few callback functions that are causing some issues at run time. Particularly, I get the following exception:
An unhandled exception of type
'System.Runtime.InteropServices.InvalidOleVariantTypeException'
occurred in ToadWrapTest.dll
Additional information: Specified OLE
variant is invalid.
On this line of code (C++/CLI):
public delegate int ManagedCallbackFunction (Object^ inst, const Object^ data);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
ManagedCallbackFunction^ m_callbackFn;
int intermidiaryCallback(void * pInstance, const void * pData)
{
void* temp = (void*)pData;
System::IntPtr ip1 = IntPtr(pInstance);
System::IntPtr ip2 = IntPtr(temp);
Object^ oInst = Marshal::GetObjectForNativeVariant(ip1);
Object^ oData = Marshal::GetObjectForNativeVariant(ip2);
//invoke the callback to c#
//return m_callbackFn::Invoke(oInst, oData);
return 0;
};
The reason I've made this "intermediary callback" was an attempt to circumvent the Invalid variant exception being thrown when I tried to directly map the delegate from C# to the native C++ code. As an attempted work-around, I declare a delegate on the C# side and pass that funcptr to the C++/CLI wrapper. I then pass the intermediary funcptr to the native C++ and just daisy chain the calls together.
What I know is that it all works in native C++ world. The problem is mapping the void* to the managed world. The following code shows the native C++ version of the callback:
int (*CallbackFunction) (void *inst, const void *data);
If anyone can help here, I'd really appreciate it.

Are pInstance and pData really VARIANT? If they are, I would expect your callback function to be more strongly typed:
int (*CallbackFunction)(VARIANT *inst, VARIANT *data);
If that's the case, in your code you should be able to look at the actual VARIANT to hand check it. If you are not really getting VARIANTs (ie, you are really just getting void * pointers), you shouldn't try to turn them into C# objects since there is no inherent meaning to them. They should get passed through as IntPtr. If you know that they should have some other type of inherent meaning, you need to marshal them as appropriate types.

Big Thanks to plinth on this one! I am posting the final solution below to anyone else who has to deal with 3rd party fun like this one! Please feel free to critique, as I am not done optimizing the code. This may still be to roundabout a solution.
First, the callback functions became:
public delegate int ManagedCallbackFunction (IntPtr oInst, IntPtr oData);
public delegate int UnManagedCallbackFunction (void* inst, const void* data);
ManagedCallbackFunction^ m_callbackFn;
Big props on this one. It just plain won't work if you try to cast from void* directly to Object^. Using the IntPtr and my intermediary callback:
int intermidiaryCallback(void * pInstance, const void * pData)
{
void* temp = (void*)pData;
return m_callbackFn->Invoke(IntPtr(pInstance), IntPtr(temp));
};
We finally get a working model on the C# side with some massaging of the objects:
public static int hReceiveTestMessage(IntPtr pInstance, IntPtr pData)
{
// provide object context for static member function
helloworld2 hw = (helloworld2)GCHandle.FromIntPtr(pInstance).Target;
if (hw == null || pData == null)
{
Console.WriteLine("hReceiveTestMessage received NULL data or instance pointer\n");
return 0;
}
// populate message with received data
IntPtr ip2 = GCHandle.ToIntPtr(GCHandle.Alloc(new DataPacketWrap(pData)));
DataPacketWrap dpw = (DataPacketWrap)GCHandle.FromIntPtr(ip2).Target;
uint retval = hw.m_testData.load_dataSets(ref dpw);
// display message contents
hw.displayTestData();
return 1;
}
I mention "massaging" the objects because the delegate is not specific to this callback function and I don't know what object pData will be until run time(from the delegates POV). Because of this issue, I have to do some extra work with the pData object. I basically had to overload the constructor in my wrapper to accept an IntPtr. Code is provided for full "clarity":
DataPacketWrap (IntPtr dp)
{
DataPacket* pdp = (DataPacket*)(dp.ToPointer());
m_NativeDataPacket = pdp;
};

Related

How to pass IntPtr to method from unmanaged C++ CLR hosting code?

I am using this tutorial as base for my code in 32bit unmanaged DLL
https://code.msdn.microsoft.com/CppHostCLR-e6581ee0
Let's say I want to call TestIntPtr
public class IntPtrTester
{
public static void TestIntPtr(IntPtr p)
{
MessageBox.Show("TestIntPtr Method was Called");
}
public static void TestInt(int p)
{
MessageBox.Show("TestInt Method was Called");
}
}
How can I pass IntPtr parameter if on C++ side it represents handle?
TestInt works, but for TestIntPtr I get the error that the method is not found. This is because the type of parameter is wrong.
In code from tutorial for TestInt I use
// HDC dc;
// The static method in the .NET class to invoke.
bstr_t bstrStaticMethodName(L"TestInt");
SAFEARRAY *psaStaticMethodArgs = NULL;
variant_t vtIntArg((INT) dc);
variant_t vtLengthRet;
...
psaStaticMethodArgs = SafeArrayCreateVector(VT_VARIANT, 0, 1);
LONG index = 0;
hr = SafeArrayPutElement(psaStaticMethodArgs, &index, &vtIntArg);
if (FAILED(hr))
{
wprintf(L"SafeArrayPutElement failed w/hr 0x%08lx\n", hr);
goto Cleanup;
}
The question is what is correct code for TestIntPtr
// The static method in the .NET class to invoke.
// HDC dc;
bstr_t bstrStaticMethodName(L"TestIntPtr");
SAFEARRAY *psaStaticMethodArgs = NULL;
variant_t vtIntArg((INT) dc); // what do I have to write here?
variant_t vtLengthRet;
I have tried:
variant_t vtIntArg((INT) dc);
variant_t vtIntArg((UINT) dc);
variant_t vtIntArg((long) dc);
variant_t vtIntArg((UINT32) dc);
variant_t vtIntArg((INT32) dc);
Maybe CLR expects IUNKNOWN of IntPtr there? But how to construct such instance? I have tried to call IntPtr constructor with this API but it returns variant of type V_INTEGER, so this is closed loop.
I know I can expose C# library using COM and how to use DllExports hack, I also can change C# part to accept just int or uint. But all these ways are not related to the question.
Currently it works for me with following C# helper
public class Helper
{
public static void help(int hdc)
{
IntPtrTester.TestIntPtr(new IntPtr(hdc));
}
}
and
variant_t vtIntArg((INT32) dc);
in c++. But this is ugly because I need this helper for the library I cannot influence.
The list of Automation compatible type is documented here: 2.2.49.3 Automation-Compatible Types
As you see, there isn't any concept of a "pointer", handle, or anything that smells "native" (low level). This is because Automation was meant originally for VB (not the .NET one, VB/VBA/VBScript, etc.) that was a language and IDE designed for ease of use, not pointer handling fun, in a time when 64-bit Windows did not existed yet.
So, IntPtr, a raw and opaque pointer (not a handle) which has the particularity to be variable in storage size depending on the executing process bitness, is not a COM automation compatible type, so it can't be put as is, as a pointer, in a VARIANT, because in a VARIANT you want to use in interop code, you can only put automation compatible things.
There are many solutions/workarounds however, because VARIANT can transport 64 bits size things, if you ask nicely. So, you could define the method like this:
public static void Test(object input)
{
// check for int (Int32) or long (Int64) here
}
And in C++ code do something like this:
variant_t vtIntArg;
if (64-bit mode)
{
vtIntArg = (__int64)dc; // force VT_I8, this overload available only if _WIN32_WINNT >= 0x0501
}
else
{
vtIntArg = (long)dc; // force VT_I4
}
Another solution is to define this in C#
public static void Test32(int ptr)
{
}
public static void Test64(long ptr)
{
}
And call the proper function, still using the __int64 overload for the Test64 method.

How to pass this pointer as context parameter for callback in c#

In C++ it's common to pass the 'this' pointer to a callback registration method so that the callback will receive this pointer back. I have a 3rd party library that i am trying to interop with from C# that uses this pattern.
Ex.
typedef HRESULT (WINAPI PFNCALLBACK*)(CALLBACKDATA *data, void *context);
HRESULT SetCallback (PFNCALLBACK *callback, void *context);
I've defined a delegate for the callback function:
public delegate int Callback(IntPtr context, CallbackData sample);
and call the SetCallback function
void SetupCallback()
{
// My stab at passing the this pointer as context for the callback
IntPtr pThis = GCHandle.ToIntPtr(GCHandle.Alloc(this));
hr = obj.SetCallback(OnCallback, pThis);
}
public static int OnCallback(CallbackData data, IntPtr context)
{
// HOW do I reconstitute the This pointer from the context parameter
}
My question is how do I pass a C# 'this' pointer to the SetCallback method and reconstitute it in the callback function?
Ugh, you are asking for an interop nightmare, especially since you are dealing with a "black box" third party DLL.
Having said that, theoretically this should be possible by utilizing the
UnmanagedFunctionPointer attribute.
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate int Callback(IntPtr context, CallbackData sample);
Then store off your delegate
static Callback _someCallBack = null;
_someCallBack += new Callback(OnCallback);
Finally pass in the delegate when calling the C code
IntPtr pThis = GCHandle.ToIntPtr(GCHandle.Alloc(this));
hr = obj.SetCallback(_someCallBack, pThis);
I also don't think you even really need to mess with the this pointer, unless you really need it in the callback honestly I would try passing in null. It's not safe in .Net land to marshall and marshall references back and re-use it.

Marshalling an array of strings from managed to native code

I have a managed function with the following declaration (both interface and implementation):
[return: MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_BSTR)]
String[] ManagedFunction()
{
String[] foo = new String[1];
foo[0] = "bar";
return foo;
}
There is also a native C++ interface with the same methods as the managed interface, inside of that interface, this method has the following declaration:
void ManagedFunction(SAFEARRAY* foo);
This function is called by native code in the following way:
void NativeFunction(ManagedBinding binding)
{
CComSafeArray<BSTR> cComSafeArray;
cComSafeArray.Create();
LPSAFEARRAY safeArray = cComSafeArray.Detach();
binding.comObject->ManagedFunction(safeArray);
}
I'm not sure what I'm doing wrong but after my managed function has been called, safeArray appears to have garbage values, somethings going wrong while marshalling the return value back to native code. Could someone with more experience than me with .Net interop please shed some light on this? Also, It might be relevant to mention that I haven't had problems with returning ValueTypes from my managed function (boolean if you're curious), something about returning a String array is messing things up. Thanks!
1) Your function return a SAFEARRAY, so why you allocate it before calling function?
2) ManagedFunction supposed to return a SAFEARRAY, so it should get a SAFEARRAY* to be able to return it! so you should say:
LPSAFEARRAY lpsa;
binding.comObject->ManagedFunction(&lpsa);
CComSafeArray<BSTR> cComSafeArray;
cComSafeArray.Attach(lpsa);
Well I finally got it to work. I created a managed representation of SAFEARRAY called ManagedSafeArray (stolen from here : http://social.msdn.microsoft.com/Forums/en-US/clr/thread/6641abfc-3a9c-4976-a523-43890b2b79a2/):
[StructLayout(LayoutKind.Sequential)]
struct ManagedSafeArray
{
public ushort dimensions; // Count of dimensions in the SAFEARRAY
public ushort features; // Flags to describe SAFEARRAY usage
public uint elementSize; // Size of an array element
public uint locks; // Number of times locked without unlocking
public IntPtr dataPtr; // Pointer to the array data
public uint elementCount; // Element count for first (only) dimension
public int lowerBound; // Lower bound for first (only) dimension
}
I changed the signature of my method to:
void ManagedMethod(ref ManagedSafeArray foo);
Inside my method, I manually updated the dataPtr field by calling Marshal.AllocCoTaskMem(...) and then copied over the strings I wanted the SAFEARRAY to contain.
I have no idea as to why the CLR wasn't able to automatically marshal the parameters to and from native code and I'd still appreciate it if someone could try to explain that.

Marshalling a char** in C#

I am interfacing with code that takes a char** (that is, a pointer to a string):
int DoSomething(Whatever* handle, char** error);
Basically, it takes a handle to its state, and if something goes wrong, it returns an error code and optionally an error message (the memory is allocated externally and freed with a second function. That part I've figued out :) ).
I, however, am unsure how to handle in in C#. What I have currently:
[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
private static unsafe extern int DoSomething(IntPtr handle, byte** error);
public static unsafe int DoSomething(IntPtr handle, out string error) {
byte* buff;
int ret = DoSomething(handle, &buff);
if(buff != 0) {
// ???
} else {
error = "";
}
return ret;
}
I've poked around, but I can't figure out how to turn that into a byte[], suitable for feeding to UTF8Encoding.UTF8.GetString()
Am I on the right track?
EDIT: To make more explicit, the library function allocates memory, which must be freed by calling another library function. If a solution does not leave me with a pointer I can free, the solution is unacceptable.
Bonus question: As implied above, this library uses UTF-8 for its strings. Do I need to do anything special in my P/Invokes, or just use string for normal const char* parameters?
You should just be able to use a ref string and have the runtime default marshaller take care of this conversion for you. You can hint the char width on the parameter with [MarshalAs(UnmanagedType.LPStr)] to make sure that you are using 8-bit characters.
Since you have a special deallocation method to call, you'll need to keep the pointer, like you've already shown in your question's example.
Here's how I'd write it:
[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
private static unsafe extern int DoSomething(
MySafeHandle handle, void** error); // byte** should work, too, I'm just lazy
Then you can get a string:
var errorMsg = Marshal.PtrToStringAnsi(new IntPtr(*error));
And cleanup:
[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
private static extern int FreeMyMemory(IntPtr h);
// ...
FreeMyMemory(new IntPtr(error));
And now we have the marshalled error, so just return it.
return errorMsg;
Also note the MySafeHandle type, which would inherit from System.Runtime.InteropServices.SafeHandle. While not strictly needed (you can use IntPtr), it gives you a better handle management when interoping with native code. Read about it here: http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.safehandle.aspx.
For reference, here is code that compiles (but, not tested yet, working on that next tested, works 100%) that does what I need. If anyone can do better, that's what I'm after :D
public static unsafe int DoSomething(IntPtr handle, out string error) {
byte* buff;
int ret = DoSomething(handle, &buff);
if(buff != null) {
int i = 0;
//count the number of bytes in the error message
while (buff[++i] != 0) ;
//allocate a managed array to store the data
byte[] tmp = new byte[i];
//(Marshal only works with IntPtrs)
IntPtr errPtr = new IntPtr(buff);
//copy the unmanaged array over
Marshal.Copy(buff, tmp, 0, i);
//get the string from the managed array
error = UTF8Encoding.UTF8.GetString(buff);
//free the unmanaged array
//omitted, since it's not important
//take a shot of whiskey
} else {
error = "";
}
return ret;
}
Edit: fixed the logic in the while loop, it had an off by one error.

C#. How to pass message from unsafe callback to managed code?

Is there a simple example of how to pass messages from unsafe callback to managed code?
I have a proprietary dll which receives some messages packed in structs and all is coming to a callback function.
The example of usage is as follows but it calls unsafe code too. I want to pass the messages into my application which is all managed code.
*P.S. I have no experience in interop or unsafe code. I used to develop in C++ 8 yrs ago but remember very little from that nightmarish times :)
P.P.S. The application is loaded as hell, the original devs claim it processes 2mil messages per sec.. I need a most efficient solution.*
static unsafe int OnCoreCallback(IntPtr pSys, IntPtr pMsg)
{
// Alias structure pointers to the pointers passed in.
CoreSystem* pCoreSys = (CoreSystem*)pSys;
CoreMessage* pCoreMsg = (CoreMessage*)pMsg;
// message handler function.
if (pCoreMsg->MessageType == Core.MSG_STATUS)
OnCoreStatus(pCoreSys, pCoreMsg);
// Continue running
return (int)Core.CALLBACKRETURN_CONTINUE;
}
Thank you.
You can use Marshal class to deal with interop code.
example:
C:
void someFunction(int msgId, void* funcCallback)
{
//do something
funcCallback(msgId); //assuming that function signature is "void func(int)"
}
C#
[DllImport("yourDllname.dll")]
static extern someFunction(int msgId, IntPtr funcCallbackPtr);
public delegate FunctionCallback(int msgId);
public FunctionCallback functionCallback;
public void SomeFunction(int msgId, out FunctionCallback functionCallback)
{
IntPtr callbackPtr;
someFunction(msgId, callbackPtr);
functionCallback = Marshal.DelegateToPointer(callbackPtr);
}
you can call as:
SomeFunction(0, (msgIdx) => Console.WriteLine("messageProcessed"));
I hope I did it right. I did not try to compile it :)

Categories

Resources