I have a program that needs to pass data from C++ to C# and back for processing. In order to do this, I have retrieved a structure, converted it into a byte array and then converted it back on the other end. However, when converting it back, the data is not correct, even though the memory dump shows that the values in memory for each variable are identical.
Here is the code to retrieve the value:
array<Byte> ^ GetPublicKeyBlob(String ^ ContainerName) {
const TCHAR * tContainer = context->marshal_as<const TCHAR*>(ContainerName);
HCRYPTPROV hProv = NULL;
CryptAcquireContext(&hProv, tContainer, MS_ENHANCED_PROV, PROV_RSA_FULL, CRYPT_MACHINE_KEYSET);
DWORD dwKeySize = 0;
CryptExportPublicKeyInfo(hProv, AT_SIGNATURE, X509_ASN_ENCODING, NULL, &dwKeySize);
PCERT_PUBLIC_KEY_INFO pbKey = (PCERT_PUBLIC_KEY_INFO)calloc(dwKeySize, sizeof(BYTE));
CryptExportPublicKeyInfo(hProv, AT_SIGNATURE, X509_ASN_ENCODING, (PCERT_PUBLIC_KEY_INFO)pbKey, &dwKeySize);
array<Byte> ^ retVal = gcnew array<Byte>(dwKeySize);
for(int i = 0; i < dwKeySize; i++)
retVal[i] = ((BYTE*)pbKey)[i];
free(pbKey);
return retVal;
}
Then on the other end, I change it back to a PCERT_PUBLIC_KEY_INFO structure with the following code:
BYTE * cpiBuffer = (BYTE*)calloc(_PublicKey->Length, sizeof(BYTE));
for(int i = 0; i < _PublicKey->Length; i++)
cpiBuffer[i] = _PublicKey[i];
PCERT_PUBLIC_KEY_INFO cpi = (PCERT_PUBLIC_KEY_INFO)cpiBuffer;
When looking at them in a memory dump, pbKey, retVal, _PublicKey, cpiBuffer and cpi all have the exact same values. But when looking at cpi as a structure, the Algorithm.pszObjId points to some erroneous memory location and when I try to use it in a function, it fails. What am I doing wrong here?
typedef struct _CRYPT_ALGORITHM_IDENTIFIER {
LPSTR pszObjId;
CRYPT_OBJID_BLOB Parameters;
} CRYPT_ALGORITHM_IDENTIFIER, *PCRYPT_ALGORITHM_IDENTIFIER;
As you can see, pszObjId is a pointer, it's contents are somewhere in memory. By casting the PCERT_PUBLIC_KEY_INFO structure to byte array you are only getting the value of the pointer, not what it points to.
On a side note, I'm not sure why are you marshaling as TCHAR*, if you want bytes then you should use char* or unsigned char*. If UNICODE is defined TCHAR will be wchar_t and that might make some difficulties.
Related
I'm marshaling some Chinese characters which have the decimal representation (utf8) as
228,184,145,230,161,148
however when I receive this in C++ I end up with the chars
-77,-13,-67,-37
I can solve this using a sbyte[] instead of string in c#, but now I'm trying to marshal a string[] so I can't use this method. Anyone have an idea as to why this is happening?
EDIT: more detailed code:
C#
[DllImport("mydll.dll",CallingConvention=CallingConvention.Cdecl)]
static extern IntPtr inputFiles(IntPtr pAlzObj, string[] filePaths, int fileNum);
string[] allfiles = Directory.GetFiles("myfolder", "*.jpg", SearchOption.AllDirectories);
string[] allFilesutf8 = allfiles.Select(i => Encoding.UTF8.GetString(Encoding.Default.GetBytes(i))).ToArray();
IntPtr pRet = inputFiles(pObj, allfiles, allfiles.Length);
C++
extern __declspec(dllexport) char* inputFiles(Alz* pObj, char** filePaths, int fileNum);
char* massAdd(Alz* pObj, char** filePaths, int fileNum)
{
if (pObj != NULL) {
try{
std::vector<const char*> imgPaths;
for (int i = 0; i < fileNum; i++)
{
char* s = *(filePaths + i);
//Here I would print out the string and the result in bytes (decimals representation) are already different.
imgPaths.push_back(s);
}
string ret = pAlzObj->myfunc(imgPaths);
const char* retTemp = ret.c_str();
char* retChar = _strdup(retTemp);
return retChar;
}
catch (const std::runtime_error& e) {
cout << "some runtime error " << e.what() << endl;
}
}
}
Also, something I found is that if I change the windows universal encoding (In language settings) to use unicode UTF-8, it works fine. Not sure why though.
When marshaling to unsigned char* (or unsigned char** as it's an array) I end up with another output, which is literally just 256+the nummbers shown when in char. 179,243,189,219. This leads me to believe there is something happening during marshaling rather than a conversion mistake on the C++ side of things.
That is because C++ strings uses standard char when stored. The char type is indeed signed and that makes those values being interpreted as negative ones.
I guess that traits may be handled inside the <xstring> header on windows (as far as I know). Specifically in:
_STD_BEGIN
template <class _Elem, class _Int_type>
struct _Char_traits { // properties of a string or stream element
using char_type = _Elem;
using int_type = _Int_type;
using pos_type = streampos;
using off_type = streamoff;
using state_type = _Mbstatet;
#if _HAS_CXX20
using comparison_category = strong_ordering;
#endif // _HAS_CXX20
I have some ideas: You solve problem by using a sbyte[] instead of string in c#, and now you are trying to marshal a string[], just use List<sbyte[]> for string array.
I am not experienced with c++ but I guess there are another libraries for strings use one of them. Look this link, link show string types can marshalling to c#. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedtype?view=net-7.0
The issue was in the marshaling. I think it was because as the data is transferred, the locale setting in the C++ dll was set to GBK (at least not UTF-8). The trick was to convert the incoming strings into UTF-8 from GBK, which I was able to do with the following function:
std::string gb_to_utf8(char* src)
{
wchar_t* strA;
int i = MultiByteToWideChar(CP_ACP, 0, src, -1, NULL, 0);
strA = (wchar_t*)malloc(i * 2);
MultiByteToWideChar(CP_ACP, 0, src, -1, strA, i);
if (!strlen((char*)strA)) {
throw std::runtime_error("error converting");
}
char utf8[1024]; //Unsure how long converted string could be, set as large number
int n = 0;
n = wcstombs(utf8, strA, sizeof(utf8));
std::string resStr = utf8;
free(strA);
return resStr;
}
Also needed to set setlocale(LC_ALL, "en_US.UTF-8"); in order for the function above to work.
I've written a basic C++ library that gets data from an OPC UA server and formats it into an array of strings (char **). I've confirmed that it works standalone, but now I'm trying to call it from a C# program using DLLs/pInvoke and running into serious memory errors.
My C# main:
List<String> resultList = new List<string>();
IntPtr inArr = new IntPtr();
inArr = Marshal.AllocHGlobal(inArr);
resultList = Utilities.ReturnStringArray(/*data*/,inArr);
C# Helper functions:
public class Utilities{
[DllImport(//DllArgs- confirmed to be correct)]
private static extern void getTopLevelNodes(/*data*/, IntPtr inArr);
public static List<String> ReturnStringArray(/*data*/,IntPtr inArr)
{
getTopLevelNodes(/*data*/,inArr); // <- this is where the AccessViolationException is thrown
//functions that convert char ** to List<String>
//return list
}
And finally, my C++ DLL implementation:
extern "C" EXPORT void getTopLevelNodes(*/data*/,char **ret){
std::vector<std::string> results = std::vector<std::string>();
//code that fills vector with strings from server
ret = (char **)realloc(ret, sizeof(char *));
ret[0] = (char *)malloc(sizeof(char));
strcpy(ret[0], "");
int count = 0;
int capacity = 1;
for (auto string : results){
ret[count] = (char*)malloc(sizeof(char) * 2048);
strcpy(ret[count++], string.c_str());
if (count == capacity){
capacity *= 2;
ret = (char **)realloc(ret, sizeof(char *)*capacity + 1);
}
}
What this should do is, initialize a List to hold the final result and IntPtr to be populated as a char ** by the C++ DLL, which is then processed back in C# and formatted into a List. However, an AccessViolationException is thrown every time I call getTopLevelNodes from C#. What can I do to fix this memory issue? Is this the best way to pass an array of strings via interop?
Thank you in advance
Edit:
I'm still looking for more answers, if there's a simpler way to implement string array interop between C# and a DLL, please, let me know!
METHOD 1 - Advanced Struct Marshalling.
As opposed to marshalling a list, try creating a c# struct like this:
[StructLayout(LayoutKind.Sequential, Pack = 2)]
public struct StringData
{
public string [] mylist; /* maybe better yet byte[][] (never tried)*/
};
Now in c# marshall like this:
IntPtr pnt = Marshal.AllocHGlobal(Marshal.SizeOf(StringData)); // Into Unmanaged space
Get A pointer to the structure.
StringData theStringData = /*get the data*/;
Marshal.StructureToPtr(theStringData, pnt, false);
// Place structure into unmanaged space.
getTopLevelNodes(/* data */, pnt); // call dll
theStringData =(StringData)Marshal.PtrToStructure(pnt,typeof(StringData));
//get structure back from unmanaged space.
Marshal.FreeHGlobal(pnt); // Free shared mem
Now in CPP:
#pragma pack(2)
/************CPP STRUCT**************/
struct StringDataCpp
{
char * strings[]
};
And the function:
extern "C" EXPORT void getTopLevelNodes(/*data*/,char *ret){ //just a byte pointer.
struct StringDataCpp *m = reinterpret_cast<struct StringDataCpp*>(ret);
//..do ur thing ..//
}
I have used this pattern with much more complicated structs as well. The key is that you're just copying byte by byte from c# and interpreting byte by byte in c++.
The 'pack' is key here, to ensure the structs align the same way in memory.
METHOD 2 - Simple byte array with fixed
//USE YOUR LIST EXCEPT List<byte>.
unsafe{
fixed (byte* cp = theStringData.ToArray)
{
getTopLevelNodes(/* data */, cp)
/////...../////
//SNIPPET TO CONVERT STRING ARRAY TO BYTE ARRAY
string[] stringlist = (/* get your strings*/);
byte[] theStringData = new stringlist [stringlist .Count()];
foreach (string b in parser)
{
// ADD SOME DELIMITER HERE FOR CPP TO SPLIT ON?
theStringData [i] = Convert.ToByte(stringlist [i]);
i++;
}
NOW
CPP just receives char*. You'll need a delimiter now to seperate the strings.
NOTE THAT YOUR STRING PROBABLY HAS DELIMETER '\0' ALREADY USE A REPLACE ALGORITHM TO REPLACE THAT WITH a ';' OR SOMETHING AND TOKENIZE EASILY IN A LOOP IN CPP USING STRTOK WITH ';' AS THE DELIMITER OR USE BOOST!
OR, try making a byte pointer array if possible.
Byte*[i] theStringStartPointers = &stringList[i]/* in a for loop*/
fixed(byte* *cp = theStringStartPointers) /// Continue
This way is much simpler. The unsafe block allows the fixed block and the fixed ensures that the c# memory management mechanism does not move that data.
How do I access a managed 2D array on the C++ side?
I know we are supposed to use pin_ptr<T> in order to access the managed array from the C++ side. It's straightforward with a 1D array, but with a 2D array I don't know how to use pin_ptr<T> correctly.
My code looks like this, where foo() will be called from the C# side:
nativeFunc(double **ptr);
foo(array<double,2> ^A)
{
const int len = A->GetLength(0);
double **ptr = (double**) alloca(sizeof(double*) * len);
for(int i = 0; i < len; i++)
{
pin_ptr<double> pinned = &A[i,0]; //Will go out of scope!
ptr[i] = pinned;
}
nativeFunc(ptr);
}
The problem is that my pin_ptr<T> will go out of scope too early, as it's located inside the loop body, and thus I think above code is NOT safe. But how can I avoid this? Apparently, it's not allowed to make an array of pin_ptr<T>, neither managed nor unmanaged. It also cannot be added to a std::vector and it cannot be made a class member either. So I'm kind of stuck here...
Thanks for any suggestions...
Okay, after some more digging, I found out that GCHandle::Alloc(x, GCHandleType::Pinned) may work as a more flexible replacement for pin_ptr<T> here. However, it seems we can only pin down the managed array as a whole. It does not seem to be possible to pin down a single sub-array (inner array) this way, like the pin_ptr<T> would do. Furthermore, by "try and error" method I have figured out that from the GCHandle handle I can get an unmanaged pointer via hdl.AddrOfPinnedObject().ToPointer() and that this one points to a continuous block of memory which contains the whole 2D array in a "flattened" (serialized) form. From here I can reconstruct the unmanaged 2D array, by using the proper base-pointer and stride. But is this considered a "safe" method and does it always work or is it even implementation specific ???
So I have hacked together a solution like this:
class ArrayPinHandlerRAII
{
public:
ArrayPinHandlerRAII(array<double,2> ^managedArray)
{
m_dimOuter = managedArray->GetLength(0);
m_dimInner = managedArray->GetLength(1);
m_handle = GCHandle::Alloc(managedArray, GCHandleType::Pinned);
m_ptr = new double*[m_dimOuter];
double *basePointer = reinterpret_cast<double*>
(m_handle.AddrOfPinnedObject().ToPointer());
for(size_t d = 0; d < m_dimOuter; d++)
{
m_ptr[d] = basePointer;
basePointer += m_dimInner;
}
}
~ArrayPinHandlerRAII(void)
{
delete [] m_ptr;
m_handle.Free();
}
inline double **data(void)
{
return m_ptr;
}
inline const size_t &dimOuter(void) const
{
return m_dimOuter;
}
inline const size_t &dimInner(void) const
{
return m_dimInner;
}
private:
GCHandle m_handle;
double **m_ptr;
size_t m_dimOuter;
size_t m_dimInner;
};
Any opinions? ;-)
Okay, one of the examples in MSDN has the following important info:
Pinning a sub-object defined in a managed object has the effect of pinning the entire object. For example, if any element of an array is pinned, then the whole array is also pinned. There are no extensions to the language for declaring a pinned array. To pin an array, declare a pinning pointer to its element type, and pin one of its elements.
So the code can actually be simplified to:
void nativeFunc(double **ptr);
void foo(array<double,2> ^A)
{
int dimOuter = managedArray->GetLength(0);
int dimInner = managedArray->GetLength(1);
pin_ptr<double> pinned = &A[i,0]; //This pins the *entire* array!
double **ptr = (double**) alloca(sizeof(double*) * dimOuter);
double *basePtr = pinned;
for(int i = 0; i < dimOuter; i++)
{
ptr[i] = basePtr;
basePtr += dimInner;
}
nativeFunc(ptr);
}
I tried to send an array from c# to c++ via com interop.
Here is the c# Code
public void SendArraytoCPlusPlus()
{
GGXForVBA.GeoAtlas GA = new GGXForVBA.GeoAtlas();
string[] arr = new string[3];
arr[0] = "One";
arr[1] = "Two";
arr[2] = "Five";
GA.GetArrayVar(arr);
}
Here is the c++ code
void GeoAtlas::GetArrayVar(VARIANT& arr)
{
AFX_MANAGE_STATE(AfxGetStaticModuleState());
SAFEARRAY* pSafeArray = arr.parray;
long lStartBound = 0;
long lEndBound = 0;
SafeArrayGetLBound(pSafeArray,1,&lStartBound);
SafeArrayGetUBound(pSafeArray,1,&lEndBound);
LPCSTR * arrayAccess = NULL;
SafeArrayAccessData( pSafeArray , (void**)&arrayAccess);
for(int iIndex = lStartBound; iIndex <= lEndBound; iIndex ++)
{
LPCTSTR myString = (LPCTSTR)arrayAccess[iIndex];
AfxMessageBox(myString);
}
}
This is the idl
[id(23)] void GetArrayVar(VARIANT arr);
The problem is, The message box only shows the FIRST letters of the strings, i.e ''O'. 'T', 'F' . I want to read the whole string. Any suggestions ?
This is based on the fact that you're sending an Unicode/UTF8 string but the message box expects an ANSI C string. Try to convert the strings to ASCII before sending them. Look at the Encoding class here http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx
You may want to try to send them as byte array.
var bytes = Encoding.ASCII.GetBytes("one");
You should pass a SAFEARRAY instead of a VARIANT:
[id(23)] void GetArrayVar([in] SAFEARRAY(BSTR) arr);
And in the implementation, access it like this:
void GeoAtlas::GetArrayVar(SAFEARRAY* arr)
{
CComSafeArray<BSTR> sa;
sa.Attach(arr);
for (int i = sa.GetLowerBound(); i <= sa.GetUpperBound(); ++i)
{
AfxMessageBox(sa[i]);
}
sa.Detach();
}
Sending side looks fine. On receiving side try this.
[id(1), helpstring("method SetArrayVar")] HRESULT SetArrayVar([in] VARIANT varMachineList);
STDMETHODIMP GeoAtlas::SetArrayVar(VARIANT arr)
{
long lower, upper;
SafeArrayGetLBound(pSafeArray, 1, &lower);
SafeArrayGetUBound(pSafeArray, 1, &upper);
DWORD dwItems = upper - lower + 1;
resize(dwItems);
SAFEARRAY* pSafeArray = arr.parray;
BSTR* pBstr;
if ( SUCCEEDED(SafeArrayAccessData(pSafeArray, (LPVOID*) &pBstr)) )
{
iterator it = begin();
for ( long i=lower; i<=upper; i++, pBstr++, it++ )
{
USES_CONVERSION;
*it = OLE2CT(*pBstr);
// Here you gets array elements and use it
}
}
// release the data block
SafeArrayUnaccessData(pSafeArray);
}
Not sure, but looks like you have Unicode/MBCS mismatch.
The managed part creates an array of UTF-16 encoded strings, while it seems that the unmanaged part is compiled with multibyte charset. Therefore, upon seeing first null character (which is actually just the second byte of UTF-16 character) your MBCS code thinks it has reached the end of the string.
One solution is to change the character set setting in your C++ project to Unicode and rebuild.
Another one is to change the code:
...
LPCWSTR * arrayAccess = NULL; // change LPCSTR -> LPCWSTR
...
for(int iIndex = lStartBound; iIndex <= lEndBound; iIndex ++)
{
bstr_t myString = arrayAccess[iIndex];
AfxMessageBox(myString);
}
bstr_t will be correctly initialized with BSTR, but it also provides conversion to const char* so call to AfxMessageBox will be OK.
I am having a really hard time getting this marshalling down.
I have umanaged code that looks like this:
WORD HLP_GetDeviceNames (LPSTR *DevNames, WORD Max_Len, WORD Max_Num)
Just FYI I did not write this unmanaged code but must use it.
Returns: WORD indicating an error.
DevNames: Pointer to an array of char arrays. Basically an array of strings that will be modified and returned back to me!
Max_Len: Length of each string (I am told this must be 256)
Max_Num: Length of array. I am using another Invoke call that is working that tells me number of devices so i know exactly how many strings to send.
I have used P/Invoke interop signatureToolkit to figure alot of this out but also read a bunch to get even further. Where I am now is here:
[DllImport("UsbMeasLib.dll")]
public static extern ushort HLP_GetDeviceNames([MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.LPStr)] ref StringBuilder[] DevNames, ushort Max_Len, ushort Max_Num);
I call my code like this:
StringBuilder[] DevNames = new StringBuilder[deviceCount];
for(int i = 0; i< deviceCount; i++)
{
DevNames[i] = new StringBuilder().Append(' ', 256);
}
HachUsbMeasLib.HLP_GetDeviceNames(ref DevNames, 256, Convert.ToUInt16(DevNames.Count()));
I am using string builder array because I need the unmanaged code to modify the string builder so that it can return the new string since string is unmutable.
When I run the code, My array is unmodified!
I'm not really sure what is going on but I think it has something to do with CLR telling unmanaged code to not modify my array in place but instead creates a new reference(pointer). Even if this is the case, I dont know how to fix it.
Thanks for any insight anybody can offer!
Try to work on low level. Declare DevNames parameter as IntPtr[]. Prepare it by the following way:
IntPtr[] devNames = new IntPtr[deviceCount];
for(int i = 0; i < deviceCount; i++)
{
devNames[i] = Marshal.AllocHGlobal[256];
}
Pass this array to HLP_GetDeviceNames. To handle output data, apply Marshal.PtrToStringAnsi to every DevNames member. Don't forget to release DevNames[i] with Marshal.FreeHGlobal in the end.
I figured this one out. Thanks to anybody who replied.
I found out how it works. I simply supply the memory space but I have to let the marshaling know that I expect in and out with this object so it allows the unmanaged code to modify the allocated space.
I did it like this:
[DllImport("UsbMeasLib.dll")]
private static extern ushort HLP_GetDeviceNames([In, Out, MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.LPStr)] string[] DevNames, ushort Max_Len, ushort Max_Num);
I use string instead of string builder because the unmanaged code will simply replace the string which is ok. I am getting the array pointer back, not modified strings. the managed code is just changing an array of pointers to point to new string objects (I think).
int numDev = HLP_GetNumDevices();
string[] names = new string[numDev];
for (int i = 0; i < names.Length; i++)
{
names[i] = new StringBuilder().Append(' ', 256).ToString();
}
ushort errorCode = HLP_GetDeviceNames(names, 256, Convert.ToUInt16(numDev));
I allocate memory for the unamanged code then let the unmanaged code chane the strings there.
This works but I dont know if I have any potential memory leaks or other potential problems.