Develop Reference

(C#) AccessViolationException when getting char ** from C++ DLL

I've written a basic C++ library that gets data from an OPC UA server and formats it into an array of strings (char **). I've confirmed that it works standalone, but now I'm trying to call it from a C# program using DLLs/pInvoke and running into serious memory errors.
My C# main:
List<String> resultList = new List<string>();
IntPtr inArr = new IntPtr();
inArr = Marshal.AllocHGlobal(inArr);
resultList = Utilities.ReturnStringArray(/*data*/,inArr);
C# Helper functions:
public class Utilities{
[DllImport(//DllArgs- confirmed to be correct)]
private static extern void getTopLevelNodes(/*data*/, IntPtr inArr);
public static List<String> ReturnStringArray(/*data*/,IntPtr inArr)
{
getTopLevelNodes(/*data*/,inArr); // <- this is where the AccessViolationException is thrown
//functions that convert char ** to List<String>
//return list
}
And finally, my C++ DLL implementation:
extern "C" EXPORT void getTopLevelNodes(*/data*/,char **ret){
std::vector<std::string> results = std::vector<std::string>();
//code that fills vector with strings from server
ret = (char **)realloc(ret, sizeof(char *));
ret[0] = (char *)malloc(sizeof(char));
strcpy(ret[0], "");
int count = 0;
int capacity = 1;
for (auto string : results){
ret[count] = (char*)malloc(sizeof(char) * 2048);
strcpy(ret[count++], string.c_str());
if (count == capacity){
capacity *= 2;
ret = (char **)realloc(ret, sizeof(char *)*capacity + 1);
}
}
What this should do is, initialize a List to hold the final result and IntPtr to be populated as a char ** by the C++ DLL, which is then processed back in C# and formatted into a List. However, an AccessViolationException is thrown every time I call getTopLevelNodes from C#. What can I do to fix this memory issue? Is this the best way to pass an array of strings via interop?
Thank you in advance
Edit:
I'm still looking for more answers, if there's a simpler way to implement string array interop between C# and a DLL, please, let me know!

METHOD 1 - Advanced Struct Marshalling.
As opposed to marshalling a list, try creating a c# struct like this:
[StructLayout(LayoutKind.Sequential, Pack = 2)]
public struct StringData
{
public string [] mylist; /* maybe better yet byte[][] (never tried)*/
};
Now in c# marshall like this:
IntPtr pnt = Marshal.AllocHGlobal(Marshal.SizeOf(StringData)); // Into Unmanaged space
Get A pointer to the structure.
StringData theStringData = /*get the data*/;
Marshal.StructureToPtr(theStringData, pnt, false);
// Place structure into unmanaged space.
getTopLevelNodes(/* data */, pnt); // call dll
theStringData =(StringData)Marshal.PtrToStructure(pnt,typeof(StringData));
//get structure back from unmanaged space.
Marshal.FreeHGlobal(pnt); // Free shared mem
Now in CPP:
#pragma pack(2)
/************CPP STRUCT**************/
struct StringDataCpp
{
char * strings[]
};
And the function:
extern "C" EXPORT void getTopLevelNodes(/*data*/,char *ret){ //just a byte pointer.
struct StringDataCpp *m = reinterpret_cast<struct StringDataCpp*>(ret);
//..do ur thing ..//
}
I have used this pattern with much more complicated structs as well. The key is that you're just copying byte by byte from c# and interpreting byte by byte in c++.
The 'pack' is key here, to ensure the structs align the same way in memory.
METHOD 2 - Simple byte array with fixed
//USE YOUR LIST EXCEPT List<byte>.
unsafe{
fixed (byte* cp = theStringData.ToArray)
{
getTopLevelNodes(/* data */, cp)
/////...../////
//SNIPPET TO CONVERT STRING ARRAY TO BYTE ARRAY
string[] stringlist = (/* get your strings*/);
byte[] theStringData = new stringlist [stringlist .Count()];
foreach (string b in parser)
{
// ADD SOME DELIMITER HERE FOR CPP TO SPLIT ON?
theStringData [i] = Convert.ToByte(stringlist [i]);
i++;
}
NOW
CPP just receives char*. You'll need a delimiter now to seperate the strings.
NOTE THAT YOUR STRING PROBABLY HAS DELIMETER '\0' ALREADY USE A REPLACE ALGORITHM TO REPLACE THAT WITH a ';' OR SOMETHING AND TOKENIZE EASILY IN A LOOP IN CPP USING STRTOK WITH ';' AS THE DELIMITER OR USE BOOST!
OR, try making a byte pointer array if possible.
Byte*[i] theStringStartPointers = &stringList[i]/* in a for loop*/
fixed(byte* *cp = theStringStartPointers) /// Continue
This way is much simpler. The unsafe block allows the fixed block and the fixed ensures that the c# memory management mechanism does not move that data.

What's the correct way to count the bytes needed for a UTF8 conversion?

I need to count the size, in bytes, that a substring will be once converted into a UTF8 byte array. This needs to happen without actually doing the conversion of that substring. The string I'm working with is very large, unfortunately, and I've got to be careful not to create another large string (or byte array) in memory.
There's a method on the Encoding.UTF8 object called GetByteCount, but I'm not seeing an overload that does it where I don't have to copy the string into a byte array. This doesn't work for me:
Encoding.UTF8.GetByteCount(stringToCount.ToCharArray(), startIndex, count);
because stringToCount.ToCharArray() will create a copy of my string.
Here's what I have right now:
public static int CalculateTotalBytesForUTF8Conversion(string stringToCount, int startIndex, int endIndex)
{
var totalBytes = 0;
for (int i = startIndex ; i < endIndex; i++)
totalBytes += Encoding.UTF8.GetByteCount(new char[] { stringToCount[i] });
return totalBytes;
}
The GetByteCount method doesn't appear to have the ability to take in just a char, so this was the compromise I'm at.
Is this the right way to determine the byte count of a substring, after conversion to UTF8, without actually doing that conversion? Or is there a better method to do this?

There doesn't appear to be a built-in method for doing this, so you could either analyze the characters yourself or do the sort of thing you're doing above. The only thing I would recommend -- reuse a char[1] array, rather than creating a new array with each iteration. Here's an extension method that jives well with the built-in methods.
public static class EncodingExtensions
{
public static int GetByteCount(this Encoding encoding, string s, int index, int count)
{
var output = 0;
var end = index + count;
var charArray = new char[1];
for (var i = index; i < end; i++)
{
charArray[0] = s[i];
output += Encoding.UTF8.GetByteCount(charArray);
}
return output;
}
}

So, there is an overload which doesn't require the caller create an array of characters first: Encoding.GetByteCount Method (Char*, Int32)
The issue is that this isn't a CLS-compliant method and will require you do some exotic coding:
public static unsafe int CalculateTotalBytesForUTF8Conversion(
string stringToCount,
int startIndex,
int endIndex)
{
// Fix the string in memory so we can grab a pointer to its location.
fixed (char* stringStart = stringToCount)
{
// Get a pointer to the start of the substring.
char* substring = stringStart + startIndex;
return Encoding.UTF8.GetByteCount(substring, endIndex - startIndex);
}
}
Key things to note here:
The method has to be marked unsafe, since we're working with pointers and direct memory manipulation.
The string is fixed for the duration of the call in order prevent the runtime moving it around - it gives us a constant location to point to, but it prevents the runtime doing memory optimization.
You should consider doing thorough performance profiling on this method to ensure it gives you a better performance profile than simply copying the string to an array.
A bit of basic profiling (a console application executing the algorithms in sequence on my desktop machine) shows that this approach executes ~35 times faster than looping over the string or converting it to a character-array.
Using pointer: ~86ms
Looping over string: ~2957ms
Converting to char array: ~3156ms
Take these figures with a pinch of salt, and also consider other factors besides just execution speed, such as long-term execution overheads (i.e. in a service process), or memory usage.

C#: How to access managed 2D array on the C++-CLI side?

How do I access a managed 2D array on the C++ side?
I know we are supposed to use pin_ptr<T> in order to access the managed array from the C++ side. It's straightforward with a 1D array, but with a 2D array I don't know how to use pin_ptr<T> correctly.
My code looks like this, where foo() will be called from the C# side:
nativeFunc(double **ptr);
foo(array<double,2> ^A)
{
const int len = A->GetLength(0);
double **ptr = (double**) alloca(sizeof(double*) * len);
for(int i = 0; i < len; i++)
{
pin_ptr<double> pinned = &A[i,0]; //Will go out of scope!
ptr[i] = pinned;
}
nativeFunc(ptr);
}
The problem is that my pin_ptr<T> will go out of scope too early, as it's located inside the loop body, and thus I think above code is NOT safe. But how can I avoid this? Apparently, it's not allowed to make an array of pin_ptr<T>, neither managed nor unmanaged. It also cannot be added to a std::vector and it cannot be made a class member either. So I'm kind of stuck here...
Thanks for any suggestions...

Okay, after some more digging, I found out that GCHandle::Alloc(x, GCHandleType::Pinned) may work as a more flexible replacement for pin_ptr<T> here. However, it seems we can only pin down the managed array as a whole. It does not seem to be possible to pin down a single sub-array (inner array) this way, like the pin_ptr<T> would do. Furthermore, by "try and error" method I have figured out that from the GCHandle handle I can get an unmanaged pointer via hdl.AddrOfPinnedObject().ToPointer() and that this one points to a continuous block of memory which contains the whole 2D array in a "flattened" (serialized) form. From here I can reconstruct the unmanaged 2D array, by using the proper base-pointer and stride. But is this considered a "safe" method and does it always work or is it even implementation specific ???
So I have hacked together a solution like this:
class ArrayPinHandlerRAII
{
public:
ArrayPinHandlerRAII(array<double,2> ^managedArray)
{
m_dimOuter = managedArray->GetLength(0);
m_dimInner = managedArray->GetLength(1);
m_handle = GCHandle::Alloc(managedArray, GCHandleType::Pinned);
m_ptr = new double*[m_dimOuter];
double *basePointer = reinterpret_cast<double*>
(m_handle.AddrOfPinnedObject().ToPointer());
for(size_t d = 0; d < m_dimOuter; d++)
{
m_ptr[d] = basePointer;
basePointer += m_dimInner;
}
}
~ArrayPinHandlerRAII(void)
{
delete [] m_ptr;
m_handle.Free();
}
inline double **data(void)
{
return m_ptr;
}
inline const size_t &dimOuter(void) const
{
return m_dimOuter;
}
inline const size_t &dimInner(void) const
{
return m_dimInner;
}
private:
GCHandle m_handle;
double **m_ptr;
size_t m_dimOuter;
size_t m_dimInner;
};
Any opinions? ;-)

Okay, one of the examples in MSDN has the following important info:
Pinning a sub-object defined in a managed object has the effect of pinning the entire object. For example, if any element of an array is pinned, then the whole array is also pinned. There are no extensions to the language for declaring a pinned array. To pin an array, declare a pinning pointer to its element type, and pin one of its elements.
So the code can actually be simplified to:
void nativeFunc(double **ptr);
void foo(array<double,2> ^A)
{
int dimOuter = managedArray->GetLength(0);
int dimInner = managedArray->GetLength(1);
pin_ptr<double> pinned = &A[i,0]; //This pins the *entire* array!
double **ptr = (double**) alloca(sizeof(double*) * dimOuter);
double *basePtr = pinned;
for(int i = 0; i < dimOuter; i++)
{
ptr[i] = basePtr;
basePtr += dimInner;
}
nativeFunc(ptr);
}

What is the fastest (possibly unsafe) way to read a byte[]?

I'm working on a server project in C#, and after a TCP message is received, it is parsed, and stored in a byte[] of exact size. (Not a buffer of fixed length, but a byte[] of an absolute length in which all data is stored.)
Now for reading this byte[] I'll be creating some wrapper functions (also for compatibility), these are the signatures of all functions I need:
public byte ReadByte();
public sbyte ReadSByte();
public short ReadShort();
public ushort ReadUShort();
public int ReadInt();
public uint ReadUInt();
public float ReadFloat();
public double ReadDouble();
public string ReadChars(int length);
public string ReadString();
The string is a \0 terminated string, and is probably encoded in ASCII or UTF-8, but I cannot tell that for sure, since I'm not writing the client.
The data exists of:
byte[] _data;
int _offset;
Now I can write all those functions manually, like this:
public byte ReadByte()
{
return _data[_offset++];
}
public sbyte ReadSByte()
{
byte r = _data[_offset++];
if (r >= 128) return (sbyte)(r - 256);
else return (sbyte)r;
}
public short ReadShort()
{
byte b1 = _data[_offset++];
byte b2 = _data[_offset++];
if (b1 >= 128) return (short)(b1 * 256 + b2 - 65536);
else return (short)(b1 * 256 + b2);
}
public short ReadUShort()
{
byte b1 = _data[_offset++];
return (short)(b1 * 256 + _data[_offset++]);
}
But I wonder if there's a faster way, not excluding the use of unsafe code, since this seems to cost too much time for simple processing.

Check out the BitConverter class, in the System namespace. It contains methods for turning parts of byte[]s into other primitive types. I've used it in similar situations and have found it suitably quick.
As for decoding strings from byte[]s, use the classes that derive from the Encoding class, in the System.Text namespace, specifically the GetString(byte[]) method (and its overloads).

One way is to map the contents of the array to a struct (providing your structure is indeed static):
http://geekswithblogs.net/taylorrich/archive/2006/08/21/88665.aspx
using System;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Sequential, Pack=1)]
struct Message
{
public int id;
[MarshalAs (UnmanagedType.ByValTStr, SizeConst=50)]
public string text;
}
void OnPacket(byte[] packet)
{
GCHandle pinnedPacket = GCHandle.Alloc(packet, GCHandleType.Pinned);
Message msg = (Message)Marshal.PtrToStructure(
pinnedPacket.AddrOfPinnedObject(),
typeof(Message));
pinnedPacket.Free();
}

You could use a BinaryReader.
A BinaryReader is a stream decorator so you would have to wrap the byte[] in a MemoryStream or attach the Reader directly to the network stream.
And then you have
int ReadInt32()
char[] ReadChars(int count)
etc.
Edit: Apparently you want 'faster execution'.
That means you are looking for an optimization in the conversion(s) from byte[], after those bytes have been received over (network) I/O.
In other words, you are trying to optimize the part that only takes up (an estimated) 0.1% of the time. Totally futile.

Unmanaged C code in C# Marshalling by ref string array!

I am having a really hard time getting this marshalling down.
I have umanaged code that looks like this:
WORD HLP_GetDeviceNames (LPSTR *DevNames, WORD Max_Len, WORD Max_Num)
Just FYI I did not write this unmanaged code but must use it.
Returns: WORD indicating an error.
DevNames: Pointer to an array of char arrays. Basically an array of strings that will be modified and returned back to me!
Max_Len: Length of each string (I am told this must be 256)
Max_Num: Length of array. I am using another Invoke call that is working that tells me number of devices so i know exactly how many strings to send.
I have used P/Invoke interop signatureToolkit to figure alot of this out but also read a bunch to get even further. Where I am now is here:
[DllImport("UsbMeasLib.dll")]
public static extern ushort HLP_GetDeviceNames([MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.LPStr)] ref StringBuilder[] DevNames, ushort Max_Len, ushort Max_Num);
I call my code like this:
StringBuilder[] DevNames = new StringBuilder[deviceCount];
for(int i = 0; i< deviceCount; i++)
{
DevNames[i] = new StringBuilder().Append(' ', 256);
}
HachUsbMeasLib.HLP_GetDeviceNames(ref DevNames, 256, Convert.ToUInt16(DevNames.Count()));
I am using string builder array because I need the unmanaged code to modify the string builder so that it can return the new string since string is unmutable.
When I run the code, My array is unmodified!
I'm not really sure what is going on but I think it has something to do with CLR telling unmanaged code to not modify my array in place but instead creates a new reference(pointer). Even if this is the case, I dont know how to fix it.
Thanks for any insight anybody can offer!

Try to work on low level. Declare DevNames parameter as IntPtr[]. Prepare it by the following way:
IntPtr[] devNames = new IntPtr[deviceCount];
for(int i = 0; i < deviceCount; i++)
{
devNames[i] = Marshal.AllocHGlobal[256];
}
Pass this array to HLP_GetDeviceNames. To handle output data, apply Marshal.PtrToStringAnsi to every DevNames member. Don't forget to release DevNames[i] with Marshal.FreeHGlobal in the end.

I figured this one out. Thanks to anybody who replied.
I found out how it works. I simply supply the memory space but I have to let the marshaling know that I expect in and out with this object so it allows the unmanaged code to modify the allocated space.
I did it like this:
[DllImport("UsbMeasLib.dll")]
private static extern ushort HLP_GetDeviceNames([In, Out, MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.LPStr)] string[] DevNames, ushort Max_Len, ushort Max_Num);
I use string instead of string builder because the unmanaged code will simply replace the string which is ok. I am getting the array pointer back, not modified strings. the managed code is just changing an array of pointers to point to new string objects (I think).
int numDev = HLP_GetNumDevices();
string[] names = new string[numDev];
for (int i = 0; i < names.Length; i++)
{
names[i] = new StringBuilder().Append(' ', 256).ToString();
}
ushort errorCode = HLP_GetDeviceNames(names, 256, Convert.ToUInt16(numDev));
I allocate memory for the unamanged code then let the unmanaged code chane the strings there.
This works but I dont know if I have any potential memory leaks or other potential problems.