I've written a basic C++ library that gets data from an OPC UA server and formats it into an array of strings (char **). I've confirmed that it works standalone, but now I'm trying to call it from a C# program using DLLs/pInvoke and running into serious memory errors.
My C# main:
List<String> resultList = new List<string>();
IntPtr inArr = new IntPtr();
inArr = Marshal.AllocHGlobal(inArr);
resultList = Utilities.ReturnStringArray(/*data*/,inArr);
C# Helper functions:
public class Utilities{
[DllImport(//DllArgs- confirmed to be correct)]
private static extern void getTopLevelNodes(/*data*/, IntPtr inArr);
public static List<String> ReturnStringArray(/*data*/,IntPtr inArr)
{
getTopLevelNodes(/*data*/,inArr); // <- this is where the AccessViolationException is thrown
//functions that convert char ** to List<String>
//return list
}
And finally, my C++ DLL implementation:
extern "C" EXPORT void getTopLevelNodes(*/data*/,char **ret){
std::vector<std::string> results = std::vector<std::string>();
//code that fills vector with strings from server
ret = (char **)realloc(ret, sizeof(char *));
ret[0] = (char *)malloc(sizeof(char));
strcpy(ret[0], "");
int count = 0;
int capacity = 1;
for (auto string : results){
ret[count] = (char*)malloc(sizeof(char) * 2048);
strcpy(ret[count++], string.c_str());
if (count == capacity){
capacity *= 2;
ret = (char **)realloc(ret, sizeof(char *)*capacity + 1);
}
}
What this should do is, initialize a List to hold the final result and IntPtr to be populated as a char ** by the C++ DLL, which is then processed back in C# and formatted into a List. However, an AccessViolationException is thrown every time I call getTopLevelNodes from C#. What can I do to fix this memory issue? Is this the best way to pass an array of strings via interop?
Thank you in advance
Edit:
I'm still looking for more answers, if there's a simpler way to implement string array interop between C# and a DLL, please, let me know!
METHOD 1 - Advanced Struct Marshalling.
As opposed to marshalling a list, try creating a c# struct like this:
[StructLayout(LayoutKind.Sequential, Pack = 2)]
public struct StringData
{
public string [] mylist; /* maybe better yet byte[][] (never tried)*/
};
Now in c# marshall like this:
IntPtr pnt = Marshal.AllocHGlobal(Marshal.SizeOf(StringData)); // Into Unmanaged space
Get A pointer to the structure.
StringData theStringData = /*get the data*/;
Marshal.StructureToPtr(theStringData, pnt, false);
// Place structure into unmanaged space.
getTopLevelNodes(/* data */, pnt); // call dll
theStringData =(StringData)Marshal.PtrToStructure(pnt,typeof(StringData));
//get structure back from unmanaged space.
Marshal.FreeHGlobal(pnt); // Free shared mem
Now in CPP:
#pragma pack(2)
/************CPP STRUCT**************/
struct StringDataCpp
{
char * strings[]
};
And the function:
extern "C" EXPORT void getTopLevelNodes(/*data*/,char *ret){ //just a byte pointer.
struct StringDataCpp *m = reinterpret_cast<struct StringDataCpp*>(ret);
//..do ur thing ..//
}
I have used this pattern with much more complicated structs as well. The key is that you're just copying byte by byte from c# and interpreting byte by byte in c++.
The 'pack' is key here, to ensure the structs align the same way in memory.
METHOD 2 - Simple byte array with fixed
//USE YOUR LIST EXCEPT List<byte>.
unsafe{
fixed (byte* cp = theStringData.ToArray)
{
getTopLevelNodes(/* data */, cp)
/////...../////
//SNIPPET TO CONVERT STRING ARRAY TO BYTE ARRAY
string[] stringlist = (/* get your strings*/);
byte[] theStringData = new stringlist [stringlist .Count()];
foreach (string b in parser)
{
// ADD SOME DELIMITER HERE FOR CPP TO SPLIT ON?
theStringData [i] = Convert.ToByte(stringlist [i]);
i++;
}
NOW
CPP just receives char*. You'll need a delimiter now to seperate the strings.
NOTE THAT YOUR STRING PROBABLY HAS DELIMETER '\0' ALREADY USE A REPLACE ALGORITHM TO REPLACE THAT WITH a ';' OR SOMETHING AND TOKENIZE EASILY IN A LOOP IN CPP USING STRTOK WITH ';' AS THE DELIMITER OR USE BOOST!
OR, try making a byte pointer array if possible.
Byte*[i] theStringStartPointers = &stringList[i]/* in a for loop*/
fixed(byte* *cp = theStringStartPointers) /// Continue
This way is much simpler. The unsafe block allows the fixed block and the fixed ensures that the c# memory management mechanism does not move that data.
Related
I'm marshaling some Chinese characters which have the decimal representation (utf8) as
228,184,145,230,161,148
however when I receive this in C++ I end up with the chars
-77,-13,-67,-37
I can solve this using a sbyte[] instead of string in c#, but now I'm trying to marshal a string[] so I can't use this method. Anyone have an idea as to why this is happening?
EDIT: more detailed code:
C#
[DllImport("mydll.dll",CallingConvention=CallingConvention.Cdecl)]
static extern IntPtr inputFiles(IntPtr pAlzObj, string[] filePaths, int fileNum);
string[] allfiles = Directory.GetFiles("myfolder", "*.jpg", SearchOption.AllDirectories);
string[] allFilesutf8 = allfiles.Select(i => Encoding.UTF8.GetString(Encoding.Default.GetBytes(i))).ToArray();
IntPtr pRet = inputFiles(pObj, allfiles, allfiles.Length);
C++
extern __declspec(dllexport) char* inputFiles(Alz* pObj, char** filePaths, int fileNum);
char* massAdd(Alz* pObj, char** filePaths, int fileNum)
{
if (pObj != NULL) {
try{
std::vector<const char*> imgPaths;
for (int i = 0; i < fileNum; i++)
{
char* s = *(filePaths + i);
//Here I would print out the string and the result in bytes (decimals representation) are already different.
imgPaths.push_back(s);
}
string ret = pAlzObj->myfunc(imgPaths);
const char* retTemp = ret.c_str();
char* retChar = _strdup(retTemp);
return retChar;
}
catch (const std::runtime_error& e) {
cout << "some runtime error " << e.what() << endl;
}
}
}
Also, something I found is that if I change the windows universal encoding (In language settings) to use unicode UTF-8, it works fine. Not sure why though.
When marshaling to unsigned char* (or unsigned char** as it's an array) I end up with another output, which is literally just 256+the nummbers shown when in char. 179,243,189,219. This leads me to believe there is something happening during marshaling rather than a conversion mistake on the C++ side of things.
That is because C++ strings uses standard char when stored. The char type is indeed signed and that makes those values being interpreted as negative ones.
I guess that traits may be handled inside the <xstring> header on windows (as far as I know). Specifically in:
_STD_BEGIN
template <class _Elem, class _Int_type>
struct _Char_traits { // properties of a string or stream element
using char_type = _Elem;
using int_type = _Int_type;
using pos_type = streampos;
using off_type = streamoff;
using state_type = _Mbstatet;
#if _HAS_CXX20
using comparison_category = strong_ordering;
#endif // _HAS_CXX20
I have some ideas: You solve problem by using a sbyte[] instead of string in c#, and now you are trying to marshal a string[], just use List<sbyte[]> for string array.
I am not experienced with c++ but I guess there are another libraries for strings use one of them. Look this link, link show string types can marshalling to c#. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedtype?view=net-7.0
The issue was in the marshaling. I think it was because as the data is transferred, the locale setting in the C++ dll was set to GBK (at least not UTF-8). The trick was to convert the incoming strings into UTF-8 from GBK, which I was able to do with the following function:
std::string gb_to_utf8(char* src)
{
wchar_t* strA;
int i = MultiByteToWideChar(CP_ACP, 0, src, -1, NULL, 0);
strA = (wchar_t*)malloc(i * 2);
MultiByteToWideChar(CP_ACP, 0, src, -1, strA, i);
if (!strlen((char*)strA)) {
throw std::runtime_error("error converting");
}
char utf8[1024]; //Unsure how long converted string could be, set as large number
int n = 0;
n = wcstombs(utf8, strA, sizeof(utf8));
std::string resStr = utf8;
free(strA);
return resStr;
}
Also needed to set setlocale(LC_ALL, "en_US.UTF-8"); in order for the function above to work.
I need to rewrite this C++ function to C#
bool DesDecrypt(const BYTE *InBuff, DWORD dwInBuffSize, BYTE *OutBuff, DWORD dwOutBuffSize, const char *TerminalID)
{
...
for(DWORD i = 0 ; i < dwInBuffSize/8 ; i++)
DES.des_ecb_encrypt((des_cblock *)InBuff+i, (des_cblock *)OutBuff+i, sched, DES_DECRYPT) ;
}
The place I am stuck is pointer arithmetic. On C++ side you can see author uses
InBuff+i
So it is advancing pointer and passing it to function.
On C# my function looks like this:
public static bool DesDecrypt(byte[] inBuff, uint inBuffSize, byte[] outBuff, uint outBufSize, string terminalID)
{
.....
}
I am stuck how to rewrite above loop(particularly how to pass pointer to next element in byte array) to C#. In C# there is no pointer arithmetic so if I do similar, it will just pass i'th value of byte array.
So how can I simulate on C# passing pointer to the next element in array ?
This is my decrypt function in C#
public static byte[] DecryptDES_ECB(byte [] ciphertext, byte [] key)
which I should use instead of C++ version: DES.des_ecb_encrypt
I am looking for such wrapper as a solution on C# side
public static byte[] DecryptDES_ECB(byte[] ciphertext, int cipherOffset, byte[] key)
{
byte [] tmp = new byte [ciphertext.Length - cipherOffset];
for(int i = 0; i<ciphertext.Length - cipherOffset; i++)
{
tmp[i] = ciphertext[cipherOffset + i];
}
return DecryptDES_ECB(tmp, key);
}
Do you think this should work? Now I will call this function on C# side in loop and pass offset as in C++.
If you use a LINQ extension and write inBuff.Skip(i) you will get an IEnumerable that yilds it's elements starting with the i inBuff element. Unless you call ToList method no copying and additional memory allocation will appear but you can treat and use your new IEnumerable like it's a subarray.
After your update:
Easiest solution would be to get some sort of subarray of your inBuff and outBuff, then perform your DecryptDES_ECB()-function and copy your results into your original arrays afterwards.
public static void DecryptDES_ECB(byte[] ciphertext, byte[] decryptedtext, int cipherOffset, byte[] key)
{
byte [] tmpCipher = new byte [ciphertext.Length - cipherOffset];
Array.copy(ciphertext, cipherOffset, tmpCipher, 0, tmpCipher.Length);
byte [] tmpDecrypt = DecryptDES_ECB(tmp, key);
Array.copy(tmpDecrypt, 0, decryptedtext, cipherOffset, tmpDecrypt.Length);
}
This method has not been tested and I don't know the underlaying library, so I can not guarantee for correctness. But generally this would be an easy (but rather slow) attempt on solving your general problem.
EDIT:
Just some additional info on Array.Copy: It performs a memmove which internally (usually) performs a call to memcpy, which is pretty damn fast. (Usually) a lot faster than your loop can possibly be.
I am writing a C# application that passes an empty string array of size 30 to a C++ DLL. This string array needs to be filled in the DLL and given back to the C# application.
I my code I observe memory corruption at the end of function call from DLL.
My C++ DLL code is as follows:
SAMPLEDLL_API BOOL InitExecution (wchar_t **paszStrings, int count)
{
for (int i = 0 ; i < count; i++)
{
mbstowcs(*(paszStrings + i), "Good",4);
//*(paszStrings + i) = "Good";
}
return TRUE;
}
My C# code is
string[] names = new[] { "Britto", "Regis" };
if (Wrapper1.InitExecution(ref names, names.Length) == 1)
MessageBox.Show("Passed");
[DllImport("MFCLibrary1.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern UInt32 InitExecution(ref string[] Names, int count);
To make this current approach work you'd need to pass StringBuilder instances rather than string. That's because the data is flowing from caller to callee. The strings are out parameters. And that means that the caller has to allocate the buffers for each string, and know how large the buffers need to be.
It's much easier to use BSTR here. This allows you to allocate the strings in the native code, and have them deallocated in the managed code. That's because BSTR is allocated on the shared COM heap, and the p/invoke marshaller understands them. Making this slight change means that the caller does not need to know how large the strings are up front.
The code would look like this:
SAMPLEDLL_API BOOL InitExecution(BSTR* names, int count)
{
for (int i = 0 ; i < count; i++)
names[i] = SysAllocStr(...);
return TRUE;
}
And on the C# side you write it like this:
[DllImport(#"mydll.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern bool InitExecution(
[Out] IntPtr[] names,
int count
);
And then you've got a bit of work to marshal from BSTR to C# string.
IntPtr[] namePtrs = new IntPtr[count];
InitExecution(namePtrs, namePtrs.Length);
string[] names = new string[namePtrs.Length];
for (int i = 0; i < namePtrs.Length; i++)
{
names[i] = Marshal.PtrToStringBSTR(namePtrs[i]);
Marshal.FreeBSTR(namePtrs[i]);
}
Actually I am using a C++ dll in C# as follows
MY C++ CODE
extern "C" __declspec(dllexport)
char** __stdcall hh()
{
static char* myArray[3] = {"A1", "BB2", "CC3",};
return myArray;
}
MY C# CODE
[DllImport(#"ourdll.dll",CharSet = CharSet.Ansi,CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr hh();
static void Main(string[] args)
{
IntPtr a = hh();
int j = 0;
string[] s=new string[100]; //I want this to be dynamic
do
{
s[j] = Marshal.PtrToStringAnsi(Marshal.ReadIntPtr(a,4*j));
j++;
}
while(s[j-1] != null);
}
I am unable to get the size of array returned by hh();How can I get the size of array so that my line of code
string[] s=new string[100];
changes to
string[] s=new string[ACtual Length of array];
You are either going to have to return the size of the array as well (possibly a variable by reference) or you can add a NULL terminator to the list. If you use a NULL terminator you can loop through the array until you find NULL in order to determine the size.
if you want to find the length of the array in PInvoke you can use Marshal.SizeOf(object) method
MSDN Link
It works for me for Structures
I am having a really hard time getting this marshalling down.
I have umanaged code that looks like this:
WORD HLP_GetDeviceNames (LPSTR *DevNames, WORD Max_Len, WORD Max_Num)
Just FYI I did not write this unmanaged code but must use it.
Returns: WORD indicating an error.
DevNames: Pointer to an array of char arrays. Basically an array of strings that will be modified and returned back to me!
Max_Len: Length of each string (I am told this must be 256)
Max_Num: Length of array. I am using another Invoke call that is working that tells me number of devices so i know exactly how many strings to send.
I have used P/Invoke interop signatureToolkit to figure alot of this out but also read a bunch to get even further. Where I am now is here:
[DllImport("UsbMeasLib.dll")]
public static extern ushort HLP_GetDeviceNames([MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.LPStr)] ref StringBuilder[] DevNames, ushort Max_Len, ushort Max_Num);
I call my code like this:
StringBuilder[] DevNames = new StringBuilder[deviceCount];
for(int i = 0; i< deviceCount; i++)
{
DevNames[i] = new StringBuilder().Append(' ', 256);
}
HachUsbMeasLib.HLP_GetDeviceNames(ref DevNames, 256, Convert.ToUInt16(DevNames.Count()));
I am using string builder array because I need the unmanaged code to modify the string builder so that it can return the new string since string is unmutable.
When I run the code, My array is unmodified!
I'm not really sure what is going on but I think it has something to do with CLR telling unmanaged code to not modify my array in place but instead creates a new reference(pointer). Even if this is the case, I dont know how to fix it.
Thanks for any insight anybody can offer!
Try to work on low level. Declare DevNames parameter as IntPtr[]. Prepare it by the following way:
IntPtr[] devNames = new IntPtr[deviceCount];
for(int i = 0; i < deviceCount; i++)
{
devNames[i] = Marshal.AllocHGlobal[256];
}
Pass this array to HLP_GetDeviceNames. To handle output data, apply Marshal.PtrToStringAnsi to every DevNames member. Don't forget to release DevNames[i] with Marshal.FreeHGlobal in the end.
I figured this one out. Thanks to anybody who replied.
I found out how it works. I simply supply the memory space but I have to let the marshaling know that I expect in and out with this object so it allows the unmanaged code to modify the allocated space.
I did it like this:
[DllImport("UsbMeasLib.dll")]
private static extern ushort HLP_GetDeviceNames([In, Out, MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.LPStr)] string[] DevNames, ushort Max_Len, ushort Max_Num);
I use string instead of string builder because the unmanaged code will simply replace the string which is ok. I am getting the array pointer back, not modified strings. the managed code is just changing an array of pointers to point to new string objects (I think).
int numDev = HLP_GetNumDevices();
string[] names = new string[numDev];
for (int i = 0; i < names.Length; i++)
{
names[i] = new StringBuilder().Append(' ', 256).ToString();
}
ushort errorCode = HLP_GetDeviceNames(names, 256, Convert.ToUInt16(numDev));
I allocate memory for the unamanged code then let the unmanaged code chane the strings there.
This works but I dont know if I have any potential memory leaks or other potential problems.