Develop Reference

Marshalling utf8 encoded chinese characters from C# to C++

I'm marshaling some Chinese characters which have the decimal representation (utf8) as
228,184,145,230,161,148
however when I receive this in C++ I end up with the chars
-77,-13,-67,-37
I can solve this using a sbyte[] instead of string in c#, but now I'm trying to marshal a string[] so I can't use this method. Anyone have an idea as to why this is happening?
EDIT: more detailed code:
C#
[DllImport("mydll.dll",CallingConvention=CallingConvention.Cdecl)]
static extern IntPtr inputFiles(IntPtr pAlzObj, string[] filePaths, int fileNum);
string[] allfiles = Directory.GetFiles("myfolder", "*.jpg", SearchOption.AllDirectories);
string[] allFilesutf8 = allfiles.Select(i => Encoding.UTF8.GetString(Encoding.Default.GetBytes(i))).ToArray();
IntPtr pRet = inputFiles(pObj, allfiles, allfiles.Length);
C++
extern __declspec(dllexport) char* inputFiles(Alz* pObj, char** filePaths, int fileNum);
char* massAdd(Alz* pObj, char** filePaths, int fileNum)
{
if (pObj != NULL) {
try{
std::vector<const char*> imgPaths;
for (int i = 0; i < fileNum; i++)
{
char* s = *(filePaths + i);
//Here I would print out the string and the result in bytes (decimals representation) are already different.
imgPaths.push_back(s);
}
string ret = pAlzObj->myfunc(imgPaths);
const char* retTemp = ret.c_str();
char* retChar = _strdup(retTemp);
return retChar;
}
catch (const std::runtime_error& e) {
cout << "some runtime error " << e.what() << endl;
}
}
}
Also, something I found is that if I change the windows universal encoding (In language settings) to use unicode UTF-8, it works fine. Not sure why though.
When marshaling to unsigned char* (or unsigned char** as it's an array) I end up with another output, which is literally just 256+the nummbers shown when in char. 179,243,189,219. This leads me to believe there is something happening during marshaling rather than a conversion mistake on the C++ side of things.

That is because C++ strings uses standard char when stored. The char type is indeed signed and that makes those values being interpreted as negative ones.
I guess that traits may be handled inside the <xstring> header on windows (as far as I know). Specifically in:
_STD_BEGIN
template <class _Elem, class _Int_type>
struct _Char_traits { // properties of a string or stream element
using char_type = _Elem;
using int_type = _Int_type;
using pos_type = streampos;
using off_type = streamoff;
using state_type = _Mbstatet;
#if _HAS_CXX20
using comparison_category = strong_ordering;
#endif // _HAS_CXX20

I have some ideas: You solve problem by using a sbyte[] instead of string in c#, and now you are trying to marshal a string[], just use List<sbyte[]> for string array.
I am not experienced with c++ but I guess there are another libraries for strings use one of them. Look this link, link show string types can marshalling to c#. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedtype?view=net-7.0

The issue was in the marshaling. I think it was because as the data is transferred, the locale setting in the C++ dll was set to GBK (at least not UTF-8). The trick was to convert the incoming strings into UTF-8 from GBK, which I was able to do with the following function:
std::string gb_to_utf8(char* src)
{
wchar_t* strA;
int i = MultiByteToWideChar(CP_ACP, 0, src, -1, NULL, 0);
strA = (wchar_t*)malloc(i * 2);
MultiByteToWideChar(CP_ACP, 0, src, -1, strA, i);
if (!strlen((char*)strA)) {
throw std::runtime_error("error converting");
}
char utf8[1024]; //Unsure how long converted string could be, set as large number
int n = 0;
n = wcstombs(utf8, strA, sizeof(utf8));
std::string resStr = utf8;
free(strA);
return resStr;
}
Also needed to set setlocale(LC_ALL, "en_US.UTF-8"); in order for the function above to work.

(C#) AccessViolationException when getting char ** from C++ DLL

I've written a basic C++ library that gets data from an OPC UA server and formats it into an array of strings (char **). I've confirmed that it works standalone, but now I'm trying to call it from a C# program using DLLs/pInvoke and running into serious memory errors.
My C# main:
List<String> resultList = new List<string>();
IntPtr inArr = new IntPtr();
inArr = Marshal.AllocHGlobal(inArr);
resultList = Utilities.ReturnStringArray(/*data*/,inArr);
C# Helper functions:
public class Utilities{
[DllImport(//DllArgs- confirmed to be correct)]
private static extern void getTopLevelNodes(/*data*/, IntPtr inArr);
public static List<String> ReturnStringArray(/*data*/,IntPtr inArr)
{
getTopLevelNodes(/*data*/,inArr); // <- this is where the AccessViolationException is thrown
//functions that convert char ** to List<String>
//return list
}
And finally, my C++ DLL implementation:
extern "C" EXPORT void getTopLevelNodes(*/data*/,char **ret){
std::vector<std::string> results = std::vector<std::string>();
//code that fills vector with strings from server
ret = (char **)realloc(ret, sizeof(char *));
ret[0] = (char *)malloc(sizeof(char));
strcpy(ret[0], "");
int count = 0;
int capacity = 1;
for (auto string : results){
ret[count] = (char*)malloc(sizeof(char) * 2048);
strcpy(ret[count++], string.c_str());
if (count == capacity){
capacity *= 2;
ret = (char **)realloc(ret, sizeof(char *)*capacity + 1);
}
}
What this should do is, initialize a List to hold the final result and IntPtr to be populated as a char ** by the C++ DLL, which is then processed back in C# and formatted into a List. However, an AccessViolationException is thrown every time I call getTopLevelNodes from C#. What can I do to fix this memory issue? Is this the best way to pass an array of strings via interop?
Thank you in advance
Edit:
I'm still looking for more answers, if there's a simpler way to implement string array interop between C# and a DLL, please, let me know!

METHOD 1 - Advanced Struct Marshalling.
As opposed to marshalling a list, try creating a c# struct like this:
[StructLayout(LayoutKind.Sequential, Pack = 2)]
public struct StringData
{
public string [] mylist; /* maybe better yet byte[][] (never tried)*/
};
Now in c# marshall like this:
IntPtr pnt = Marshal.AllocHGlobal(Marshal.SizeOf(StringData)); // Into Unmanaged space
Get A pointer to the structure.
StringData theStringData = /*get the data*/;
Marshal.StructureToPtr(theStringData, pnt, false);
// Place structure into unmanaged space.
getTopLevelNodes(/* data */, pnt); // call dll
theStringData =(StringData)Marshal.PtrToStructure(pnt,typeof(StringData));
//get structure back from unmanaged space.
Marshal.FreeHGlobal(pnt); // Free shared mem
Now in CPP:
#pragma pack(2)
/************CPP STRUCT**************/
struct StringDataCpp
{
char * strings[]
};
And the function:
extern "C" EXPORT void getTopLevelNodes(/*data*/,char *ret){ //just a byte pointer.
struct StringDataCpp *m = reinterpret_cast<struct StringDataCpp*>(ret);
//..do ur thing ..//
}
I have used this pattern with much more complicated structs as well. The key is that you're just copying byte by byte from c# and interpreting byte by byte in c++.
The 'pack' is key here, to ensure the structs align the same way in memory.
METHOD 2 - Simple byte array with fixed
//USE YOUR LIST EXCEPT List<byte>.
unsafe{
fixed (byte* cp = theStringData.ToArray)
{
getTopLevelNodes(/* data */, cp)
/////...../////
//SNIPPET TO CONVERT STRING ARRAY TO BYTE ARRAY
string[] stringlist = (/* get your strings*/);
byte[] theStringData = new stringlist [stringlist .Count()];
foreach (string b in parser)
{
// ADD SOME DELIMITER HERE FOR CPP TO SPLIT ON?
theStringData [i] = Convert.ToByte(stringlist [i]);
i++;
}
NOW
CPP just receives char*. You'll need a delimiter now to seperate the strings.
NOTE THAT YOUR STRING PROBABLY HAS DELIMETER '\0' ALREADY USE A REPLACE ALGORITHM TO REPLACE THAT WITH a ';' OR SOMETHING AND TOKENIZE EASILY IN A LOOP IN CPP USING STRTOK WITH ';' AS THE DELIMITER OR USE BOOST!
OR, try making a byte pointer array if possible.
Byte*[i] theStringStartPointers = &stringList[i]/* in a for loop*/
fixed(byte* *cp = theStringStartPointers) /// Continue
This way is much simpler. The unsafe block allows the fixed block and the fixed ensures that the c# memory management mechanism does not move that data.

C#: How to access managed 2D array on the C++-CLI side?

How do I access a managed 2D array on the C++ side?
I know we are supposed to use pin_ptr<T> in order to access the managed array from the C++ side. It's straightforward with a 1D array, but with a 2D array I don't know how to use pin_ptr<T> correctly.
My code looks like this, where foo() will be called from the C# side:
nativeFunc(double **ptr);
foo(array<double,2> ^A)
{
const int len = A->GetLength(0);
double **ptr = (double**) alloca(sizeof(double*) * len);
for(int i = 0; i < len; i++)
{
pin_ptr<double> pinned = &A[i,0]; //Will go out of scope!
ptr[i] = pinned;
}
nativeFunc(ptr);
}
The problem is that my pin_ptr<T> will go out of scope too early, as it's located inside the loop body, and thus I think above code is NOT safe. But how can I avoid this? Apparently, it's not allowed to make an array of pin_ptr<T>, neither managed nor unmanaged. It also cannot be added to a std::vector and it cannot be made a class member either. So I'm kind of stuck here...
Thanks for any suggestions...

Okay, after some more digging, I found out that GCHandle::Alloc(x, GCHandleType::Pinned) may work as a more flexible replacement for pin_ptr<T> here. However, it seems we can only pin down the managed array as a whole. It does not seem to be possible to pin down a single sub-array (inner array) this way, like the pin_ptr<T> would do. Furthermore, by "try and error" method I have figured out that from the GCHandle handle I can get an unmanaged pointer via hdl.AddrOfPinnedObject().ToPointer() and that this one points to a continuous block of memory which contains the whole 2D array in a "flattened" (serialized) form. From here I can reconstruct the unmanaged 2D array, by using the proper base-pointer and stride. But is this considered a "safe" method and does it always work or is it even implementation specific ???
So I have hacked together a solution like this:
class ArrayPinHandlerRAII
{
public:
ArrayPinHandlerRAII(array<double,2> ^managedArray)
{
m_dimOuter = managedArray->GetLength(0);
m_dimInner = managedArray->GetLength(1);
m_handle = GCHandle::Alloc(managedArray, GCHandleType::Pinned);
m_ptr = new double*[m_dimOuter];
double *basePointer = reinterpret_cast<double*>
(m_handle.AddrOfPinnedObject().ToPointer());
for(size_t d = 0; d < m_dimOuter; d++)
{
m_ptr[d] = basePointer;
basePointer += m_dimInner;
}
}
~ArrayPinHandlerRAII(void)
{
delete [] m_ptr;
m_handle.Free();
}
inline double **data(void)
{
return m_ptr;
}
inline const size_t &dimOuter(void) const
{
return m_dimOuter;
}
inline const size_t &dimInner(void) const
{
return m_dimInner;
}
private:
GCHandle m_handle;
double **m_ptr;
size_t m_dimOuter;
size_t m_dimInner;
};
Any opinions? ;-)

Okay, one of the examples in MSDN has the following important info:
Pinning a sub-object defined in a managed object has the effect of pinning the entire object. For example, if any element of an array is pinned, then the whole array is also pinned. There are no extensions to the language for declaring a pinned array. To pin an array, declare a pinning pointer to its element type, and pin one of its elements.
So the code can actually be simplified to:
void nativeFunc(double **ptr);
void foo(array<double,2> ^A)
{
int dimOuter = managedArray->GetLength(0);
int dimInner = managedArray->GetLength(1);
pin_ptr<double> pinned = &A[i,0]; //This pins the *entire* array!
double **ptr = (double**) alloca(sizeof(double*) * dimOuter);
double *basePtr = pinned;
for(int i = 0; i < dimOuter; i++)
{
ptr[i] = basePtr;
basePtr += dimInner;
}
nativeFunc(ptr);
}

passing char pointer array of c++ dLL to c# string

consider the following c++ code
#include "stdafx.h"
#include<iostream>
using namespace std;
this much part i want in c#..
void ping(int,char* d[]);
void ping(int a,char *b[])
{
int size;
size=sizeof(b)/sizeof(int); // total size of array/size of array data type
//cout<<size;
for(int i=0;i<=size;i++)
cout<<"ping "<<a<<b[i]<<endl;
}
and below part is in c++
int _tmain(int argc, _TCHAR* argv[])
{
void (*funcptr)(int,char* d[]);
char* c[]={"a","b"};
funcptr= ping;
funcptr(10,c);
return 0;
}
how can i implement the same in c#..
m new to c#. how can i have char pointer array in c#?

You usually avoid char* or char[] in favor of the string class. Rather than having a char* d[], you would have a string[] d instead, if you want an array of strings, or a simple string d if you want a single list of characters.
Interop between C++ and C# is always tricky. Some good references include Pass C# string to C++ and pass C++ result (string, char*.. whatever) to C# and Using arrays and pointers in C# with C DLL.

A string is a list of characters. With your mention of character manipulation and your use of loops I'm assuming your concern is with targetting particular characters from one list/array - and in this sense you can code almost identically when interrogating particular characters from a string (as if it were a char array).
For example:
string testString = "hello";
char testChar = testString[2];
testChar, in this case, will be equal to 'l'.

Firstly, your "C++" code is actually C and bad C at that- it won't execute correctly at all. sizeof(b) will not give you the size of the array or anything like it, it will give you the size of a pointer. Consider writing some correct C++ before attempting to convert to C#.
template<int N> void ping(int a, std::array<std::string, N>& arr) {
for(int i = 0; i < N; i++) std::cout << a << arr[i] << "\n";
}
int _tmain(int argc, _TCHAR* argv[]) {
std::array<std::string, 2> c = { "a", "b" };
ping(10, c);
return 0;
}
C# doesn't have statically sized arrays, but the rest is easily done
public static void ping(int a, string[] arr) {
for(int i = 0; i < arr.Length; i++) {
System.Console.Write(a);
System.Console.Write(arr[i]);
}
}
public static void Main(string[] args) {
string[] arr = { "a", "b" };
ping(10, arr);
}

This should help you, although note that the size of the output buffer is fixed so this won't work for dynamic sized strings, you need to know the size beforehand.
public unsafe static void Foo(char*[] input)
{
foreach(var cptr in input)
{
IntPtr ptr = new IntPtr(cptr);
char[] output = new char[5]; //NOTE: Size is fixed
Marshal.Copy(ptr, output, 0, output.Length);
Console.WriteLine(new string(output));
}
}
Remember to allow unsafe code, since that's the only way you can use fixed pointers in C# (Right-click project, properties, build, allow unsafe code).
Next time be more specific and clear, and try to act more respectfully towards people here, we're not getting paid to help you you know :-)

we can do it as
in DLL we will have
extern "C" __declspec(dllexport) void __stdcall Caller()
{
static char* myArray[3];
myArray[0]="aasdasdasdasdas8888";
myArray[1]="sssss";
FunctionPtr1(2,myArray);
}
and in C# i just added following lines
public static void ping(int a, IntPtr x)
{
Console.WriteLine("Entered Ping Command");
//code to fetch char pointer array from DLL starts here
IntPtr c = x;
int j = 0;
string[] s = new string[100]; //I want this to be dynamic
Console.WriteLine("content of array");
Console.WriteLine("");
do
{
s[j] = Marshal.PtrToStringAnsi(Marshal.ReadIntPtr(c, 4 * j));
Console.WriteLine(s[j]);
j++;
} while (s[j - 1] != null);
//**********end****************
Console.WriteLine("Integer value received from DLL is "+a);
}

Passing a structure appears to corrupt data

I have a program that needs to pass data from C++ to C# and back for processing. In order to do this, I have retrieved a structure, converted it into a byte array and then converted it back on the other end. However, when converting it back, the data is not correct, even though the memory dump shows that the values in memory for each variable are identical.
Here is the code to retrieve the value:
array<Byte> ^ GetPublicKeyBlob(String ^ ContainerName) {
const TCHAR * tContainer = context->marshal_as<const TCHAR*>(ContainerName);
HCRYPTPROV hProv = NULL;
CryptAcquireContext(&hProv, tContainer, MS_ENHANCED_PROV, PROV_RSA_FULL, CRYPT_MACHINE_KEYSET);
DWORD dwKeySize = 0;
CryptExportPublicKeyInfo(hProv, AT_SIGNATURE, X509_ASN_ENCODING, NULL, &dwKeySize);
PCERT_PUBLIC_KEY_INFO pbKey = (PCERT_PUBLIC_KEY_INFO)calloc(dwKeySize, sizeof(BYTE));
CryptExportPublicKeyInfo(hProv, AT_SIGNATURE, X509_ASN_ENCODING, (PCERT_PUBLIC_KEY_INFO)pbKey, &dwKeySize);
array<Byte> ^ retVal = gcnew array<Byte>(dwKeySize);
for(int i = 0; i < dwKeySize; i++)
retVal[i] = ((BYTE*)pbKey)[i];
free(pbKey);
return retVal;
}
Then on the other end, I change it back to a PCERT_PUBLIC_KEY_INFO structure with the following code:
BYTE * cpiBuffer = (BYTE*)calloc(_PublicKey->Length, sizeof(BYTE));
for(int i = 0; i < _PublicKey->Length; i++)
cpiBuffer[i] = _PublicKey[i];
PCERT_PUBLIC_KEY_INFO cpi = (PCERT_PUBLIC_KEY_INFO)cpiBuffer;
When looking at them in a memory dump, pbKey, retVal, _PublicKey, cpiBuffer and cpi all have the exact same values. But when looking at cpi as a structure, the Algorithm.pszObjId points to some erroneous memory location and when I try to use it in a function, it fails. What am I doing wrong here?

typedef struct _CRYPT_ALGORITHM_IDENTIFIER {
LPSTR pszObjId;
CRYPT_OBJID_BLOB Parameters;
} CRYPT_ALGORITHM_IDENTIFIER, *PCRYPT_ALGORITHM_IDENTIFIER;
As you can see, pszObjId is a pointer, it's contents are somewhere in memory. By casting the PCERT_PUBLIC_KEY_INFO structure to byte array you are only getting the value of the pointer, not what it points to.
On a side note, I'm not sure why are you marshaling as TCHAR*, if you want bytes then you should use char* or unsigned char*. If UNICODE is defined TCHAR will be wchar_t and that might make some difficulties.