I want to define a callback function in my C# code and pass it to some native C++ code, then have the C++ code invoke it later. The callback needs to receive a variable-length array of structs, with each struct containing some array fields.
I have had success passing a single struct with an array field, and a variable-length array of structs with scalar fields, but not a variable-length array of structs with array fields.
Here's my C# code. I'm omitting the code that registers the C# callback method with the C++ code, as I don't think that's the issue; it works fine except for the specific problematic case.
The struct:
[StructLayout(LayoutKind.Sequential)]
public struct Foo
{
[MarshalAs(UnmanagedType.ByValArray, ArraySubType=UnmanagedType.R4, SizeConst = 2)]
public float[] a;
}
The callback declaration
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void Callback(Int32 count,
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1)] Foo[] foos);
The callback method itself
public void onFoo(Int32 count, Foo[] foos) {
Debug.Log("in onFoo callback, foo values=" + foo[0].a + ", " + foo[1].a);
}
And here's the C++ code:
First the struct:
typedef struct {
float a[2];
} Foo;
and the callback invocation:
Foo* foos = new Foo[2];
foos[0].a[0] = 1.11;
foos[0].a[1] = 2.22;
foos[1].a[0] = 3.33;
foos[1].a[1] = 4.44;
onFoo(2, foos);
delete[] foos;
For the problematic case, my callback method is not being invoked (I get no log output). I've done quite a lot of Googling but haven't found anything covering this particular scenario. Do I need a custom marshaler?
This isn't an answer, but an observation of your callback invocation. It's been many years since I've done C++, but shouldn't the following
Foo* foos = new Foo[2];
foos[0].a = 1.11;
foos[1].a = 2.22;
onFoo(2, foos);
Be
Foo* foos = new Foo[2];
foos[0].a[0] = 1.11;
foos[0].a[1] = 1.12;
foos[1].a[0] = 2.22;
foos[1].a[1] = 2.23;
onFoo(2, foos);
OK, answering my own question here with an alternative approach that works. I'll leave it unaccepted in case someone else comes along and solves the originally stated problem.
The workaround is to pass an array of struct pointers to the callback instead of an array of structs. The callback signature changes to
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate void Callback(Int32 count,
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1)] IntPtr[] fooPtrs);
and in the C++ code, I allocate an array of Foo*, then allocate and initialize each Foo separately, storing its pointer into the array.
Foo** foos = new Foo*[2];
Foo* foo1 = new Foo();
foo1->a[0] = 1.11;
foo1->a[1] = 2.22;
foos[0] = foo1;
Foo* foo2 = new Foo();
foo2->a[0] = 3.33;
foo2->a[1] = 4.44;
foos[1] = foo2;
onFoo(2, foos);
delete foo1;
delete foo2;
delete[] foos;
Back on the C# side, in the callback method, I allocate a Foo array, loop through the incoming IntPtr array, and marshal each element using Marshal.PtrToStructure, like so:
public void onFoo(Int32 count, IntPtr[] FooPtrs)
{
Foo[] foos = new Foo[count];
for (int i=0; i< count; i++)
{
foos[i] = (Foo)Marshal.PtrToStructure(ptrs[i], typeof(Foo));
}
}
The downside to this approach is that there is more memory allocation and copying involved, rather than being able to map the C# struct array directly onto the memory allocated in C++.
Related
I know this can be done by mallocing in C, passing malloced pointer to delegate with parameter type IntPtr, marshalling to string[] and then freeing malloced memory with separate, exported C-function from managed code.
My question is: Can this be done simpler way? E.g. :
C# delegate parameter is of type string[]?
no separate free function to call from managed code
EDIT: I tried with delegate signature:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
MyManagedDelegate(string[] values, int valueCount)
and fucntion in C:
void NativeCallDelegate(char *pStringValues[], int nValues)
{
if (gSetStringValuesCB)
gSetStringValuesCB(pStringValues, nValues);
}
calling it in C:
char *Values[]= {"One", "Two", "Three"};
NativeCallDelegate(Values, 3);
This results in that i can use only 1st string in array.
Here's how to do it properly, I'll give a full example so it's reproductible.
The C side
typedef void(*setStringValuesCB_t)(char *pStringValues[], int nValues);
static setStringValuesCB_t gSetStringValuesCB;
void NativeCallDelegate(char *pStringValues[], int nValues)
{
if (gSetStringValuesCB)
gSetStringValuesCB(pStringValues, nValues);
}
__declspec(dllexport) void NativeLibCall(setStringValuesCB_t callback)
{
gSetStringValuesCB = callback;
char *Values[] = { "One", "Two", "Three" };
NativeCallDelegate(Values, 3);
}
Nothing fancy here, I just added the necessary glue code and left the rest alone.
The C# side
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void MyManagedDelegate(
[MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr, SizeParamIndex = 1)]
string[] values,
int valueCount);
[DllImport("NativeTemp", CallingConvention = CallingConvention.Cdecl)]
public static extern void NativeLibCall(MyManagedDelegate callback);
public static void Main()
{
NativeLibCall(PrintReceivedData);
}
public static void PrintReceivedData(string[] values, int valueCount)
{
foreach (var item in values)
Console.WriteLine(item);
}
The trick lies in the marshaling part:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate void MyManagedDelegate(
[MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr, SizeParamIndex = 1)]
string[] values,
int valueCount);
The MarshalAs attribute tells the .NET marshaler the following:
UnmanagedType.LPArray You're getting an array...
ArraySubType = UnmanagedType.LPStr ...of standard C strings...
SizeParamIndex = 1 ...and the size of that array is specified by the second parameter.
The C strings are copied and converted to System.String instances by the .NET marshaler before the invocation of your C# method. So if you need to pass dynamically generated strings to C#, you malloc them, then you call gSetStringValuesCB, and you can free them immediately afterwards, all from the C code, as .NET has its own copy of the data.
You can refer to the docs:
UnmanagedType.LPArray:
A pointer to the first element of a C-style array. When marshaling from managed to unmanaged code, the length of the array is determined by the length of the managed array. When marshaling from unmanaged to managed code, the length of the array is determined from the MarshalAsAttribute.SizeConst and MarshalAsAttribute.SizeParamIndex fields, optionally followed by the unmanaged type of the elements within the array when it is necessary to differentiate among string types.
UnmanagedType.LPStr:
A single byte, null-terminated ANSI character string. You can use this member on the System.String and System.Text.StringBuilder data types.
MarshalAs.SizeParamIndex:
Indicates the zero-based parameter that contains the count of array elements, similar to size_is in COM.
I came up with far from optimal solution:
public delegate void MyManagedDelegate([MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr, SizeConst=10)]string[] values, int valueCount);
This is not working if called like this:
char *Values[]= {"One", "Two", "Three"};
NativeCallDelegate(Values, 3);
I could have fixed size 10 array where the values are copied and that is always passed to delegate. This is not what I want. I wonder if there is good solution to this...
How do I marshal this C++ type?
The ABS_DATA structure is used to associate an arbitrarily long data block with the length information. The declared length of the Data array is 1, but the actual length is given by the Length member.
typedef struct abs_data {
ABS_DWORD Length;
ABS_BYTE Data[ABS_VARLEN];
} ABS_DATA;
I tried the following code, but it's not working. The data variable is always empty and I'm sure it has data in there.
[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential, CharSet = System.Runtime.InteropServices.CharSet.Ansi)]
public struct abs_data
{
/// ABS_DWORD->unsigned int
public uint Length;
/// ABS_BYTE[1]
[System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst = 1)]
public string Data;
}
Old question, but I recently had to do this myself and all the existing answers are poor, so...
The best solution for marshaling a variable-length array in a struct is to use a custom marshaler. This lets you control the code that the runtime uses to convert between managed and unmanaged data. Unfortunately, custom marshaling is poorly-documented and has a few bizarre limitations. I'll cover those quickly, then go over the solution.
Annoyingly, you can't use custom marshaling on an array element of a struct or class. There's no documented or logical reason for this limitation, and the compiler won't complain, but you'll get an exception at runtime. Also, there's a function that custom marshalers must implement, int GetNativeDataSize(), which is obviously impossible to implement accurately (it doesn't pass you an instance of the object to ask its size, so you can only go off the type, which is of course variable size!) Fortunately, this function doesn't matter. I've never seen it get called, and it the custom marshaler works fine even if it returns a bogus value (one MSDN example has it return -1).
First of all, here's what I think your native prototype might look like (I'm using P/Invoke here, but it works for COM too):
// Unmanaged C/C++ code prototype (guess)
//void DoThing (ABS_DATA *pData);
// Guess at your managed call with the "marshal one-byte ByValArray" version
//[DllImport("libname.dll")] public extern void DoThing (ref abs_data pData);
Here's the naïve version of how you might have used a custom marshaler (which really ought to have worked). I'll get to the marshaler itself in a bit...
[StructLayout(LayoutKind.Sequential)]
public struct abs_data
{
// Don't need the length as a separate filed; managed arrays know it.
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<byte>))]
public byte[] Data;
}
// Now you can just pass the struct but it takes arbitrary sizes!
[DllImport("libname.dll")] public extern void DoThing (ref abs_data pData);
Unfortunately, at runtime, you apparently can't marshal arrays inside data structures as anything except SafeArray or ByValArray. SafeArrays are counted, but they look nothing like the (extremely common) format that you're looking for here. So that won't work. ByValArray, of course, requires that the length be known at compile time, so that doesn't work either (as you ran into). Bizarrely, though, you can use custom marshaling on array parameters, This is annoying because you have to put the MarshalAsAttribute on every parameter that uses this type, instead of just putting it on one field and having that apply everywhere you use the type containing that field, but c'est la vie. It looks like this:
[StructLayout(LayoutKind.Sequential)]
public struct abs_data
{
// Don't need the length as a separate filed; managed arrays know it.
// This isn't an array anymore; we pass an array of this instead.
public byte Data;
}
// Now you pass an arbitrary-sized array of the struct
[DllImport("libname.dll")] public extern void DoThing (
// Have to put this huge stupid attribute on every parameter of this type
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<abs_data>))]
// Don't need to use "ref" anymore; arrays are ref types and pass as pointer-to
abs_data[] pData);
In that example, I preserved the abs_data type, in case you want to do something special with it (constructors, static functions, properties, inheritance, whatever). If your array elements consisted of a complex type, you would modify the struct to represent that complex type. However, in this case, abs_data is basically just a renamed byte - it's not even "wrapping" the byte; as far as the native code is concerned it's more like a typedef - so you can just pass an array of bytes and skip the struct entirely:
// Actually, you can just pass an arbitrary-length byte array!
[DllImport("libname.dll")] public extern void DoThing (
// Have to put this huge stupid attribute on every parameter of this type
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<byte>))]
byte[] pData);
OK, so now you can see how to declare the array element type (if needed), and how to pass the array to an unmanaged function. However, we still need that custom marshaler. You should read "Implementing the ICustomMarshaler Interface" but I'll cover this here, with inline comments. Note that I use some shorthand conventions (like Marshal.SizeOf<T>()) that require .NET 4.5.1 or higher.
// The class that does the marshaling. Making it generic is not required, but
// will make it easier to use the same custom marshaler for multiple array types.
public class ArrayMarshaler<T> : ICustomMarshaler
{
// All custom marshalers require a static factory method with this signature.
public static ICustomMarshaler GetInstance (String cookie)
{
return new ArrayMarshaler<T>();
}
// This is the function that builds the managed type - in this case, the managed
// array - from a pointer. You can just return null here if only sending the
// array as an in-parameter.
public Object MarshalNativeToManaged (IntPtr pNativeData)
{
// First, sanity check...
if (IntPtr.Zero == pNativeData) return null;
// Start by reading the size of the array ("Length" from your ABS_DATA struct)
int length = Marshal.ReadInt32(pNativeData);
// Create the managed array that will be returned
T[] array = new T[length];
// For efficiency, only compute the element size once
int elSiz = Marshal.SizeOf<T>();
// Populate the array
for (int i = 0; i < length; i++)
{
array[i] = Marshal.PtrToStructure<T>(pNativeData + sizeof(int) + (elSiz * i));
}
// Alternate method, for arrays of primitive types only:
// Marshal.Copy(pNativeData + sizeof(int), array, 0, length);
return array;
}
// This is the function that marshals your managed array to unmanaged memory.
// If you only ever marshal the array out, not in, you can return IntPtr.Zero
public IntPtr MarshalManagedToNative (Object ManagedObject)
{
if (null == ManagedObject) return IntPtr.Zero;
T[] array = (T[])ManagedObj;
int elSiz = Marshal.SizeOf<T>();
// Get the total size of unmanaged memory that is needed (length + elements)
int size = sizeof(int) + (elSiz * array.Length);
// Allocate unmanaged space. For COM, use Marshal.AllocCoTaskMem instead.
IntPtr ptr = Marshal.AllocHGlobal(size);
// Write the "Length" field first
Marshal.WriteInt32(ptr, array.Length);
// Write the array data
for (int i = 0; i < array.Length; i++)
{ // Newly-allocated space has no existing object, so the last param is false
Marshal.StructureToPtr<T>(array[i], ptr + sizeof(int) + (elSiz * i), false);
}
// If you're only using arrays of primitive types, you could use this instead:
//Marshal.Copy(array, 0, ptr + sizeof(int), array.Length);
return ptr;
}
// This function is called after completing the call that required marshaling to
// unmanaged memory. You should use it to free any unmanaged memory you allocated.
// If you never consume unmanaged memory or other resources, do nothing here.
public void CleanUpNativeData (IntPtr pNativeData)
{
// Free the unmanaged memory. Use Marshal.FreeCoTaskMem if using COM.
Marshal.FreeHGlobal(pNativeData);
}
// If, after marshaling from unmanaged to managed, you have anything that needs
// to be taken care of when you're done with the object, put it here. Garbage
// collection will free the managed object, so I've left this function empty.
public void CleanUpManagedData (Object ManagedObj)
{ }
// This function is a lie. It looks like it should be impossible to get the right
// value - the whole problem is that the size of each array is variable!
// - but in practice the runtime doesn't rely on this and may not even call it.
// The MSDN example returns -1; I'll try to be a little more realistic.
public int GetNativeDataSize ()
{
return sizeof(int) + Marshal.SizeOf<T>();
}
}
Whew, that was long! Well, there you have it. I hope people see this, because there's a lot of bad answers and misunderstanding out there...
It is not possible to marshal structs containing variable-length arrays (but it is possible to marshal variable-length arrays as function parameters). You will have to read your data manually:
IntPtr nativeData = ... ;
var length = Marshal.ReadUInt32 (nativeData) ;
var bytes = new byte[length] ;
Marshal.Copy (new IntPtr ((long)nativeData + 4), bytes, 0, length) ;
If the data being saved isn't a string, you don't have to store it in a string. I usually do not marshal to a string unless the original data type was a char*. Otherwise a byte[] should do.
Try:
[MarshalAs(UnmanagedType.ByValArray, SizeConst=[whatever your size is]]
byte[] Data;
If you need to convert this to a string later, use:
System.Text.Encoding.UTF8.GetString(your byte array here).
Obviously, you need to vary the encoding to what you need, though UTF-8 usually is sufficient.
I see the problem now, you have to marshal a VARIABLE length array. The MarshalAs does not allow this and the array will have to be sent by reference.
If the array length is variable, your byte[] needs to be an IntPtr, so you would use,
IntPtr Data;
Instead of
[MarshalAs(UnmanagedType.ByValArray, SizeConst=[whatever your size is]]
byte[] Data;
You can then use the Marshal class to access the underlying data.
Something like:
uint length = yourABSObject.Length;
byte[] buffer = new byte[length];
Marshal.Copy(buffer, 0, yourABSObject.Data, length);
You may need to clean up your memory when you are finished to avoid a leak, though I suspect the GC will clean it up when yourABSObject goes out of scope. Anyway, here is the cleanup code:
Marshal.FreeHGlobal(yourABSObject.Data);
You are trying to marshal something that is a byte[ABS_VARLEN] as if it were a string of length 1. You'll need to figure out what the ABS_VARLEN constant is and marshal the array as:
[MarshalAs(UnmanagedType.LPArray, SizeConst = 1024)]
public byte[] Data;
(The 1024 there is a placeholder; fill in whatever the actual value of ASB_VARLEN is.)
In my opinion, it's simpler and more efficient to pin the array and take its address.
Assuming you need to pass abs_data to myNativeFunction(abs_data*):
public struct abs_data
{
public uint Length;
public IntPtr Data;
}
[DllImport("myDll.dll")]
static extern void myNativeFunction(ref abs_data data);
void CallNativeFunc(byte[] data)
{
GCHandle pin = GCHandle.Alloc(data, GCHandleType.Pinned);
abs_data tmp;
tmp.Length = data.Length;
tmp.Data = pin.AddrOfPinnedObject();
myNativeFunction(ref tmp);
pin.Free();
}
I have a class definition as follows:
[StructLayout(LayoutKind.Sequential)]
public class OUR_MEM_STR
{
public byte[] p;
public int len;
};
This is an equivalent defintion of the C structure below:
typedef struct
{
void *p;
int len;
} OUR_MEM_STR;
I used byte[] instead of IntPtr type for member p becuase of the way it was being used thorughout c# project.
I have defined an object obj with len = 10 and p = new byte[10]
I want to make it an intptr. How do I get the size of the object for that?
IntPtr pObj = Marshal.AllocHGlobal(obj.len + sizeof(int));
Marshal.StructureToPtr(obj, pObj, true);
See what I did there. It seems too hard coded. If I do the below snippet;
IntPtr pObj = Marshal.AllocHGlobal(Marshal.SizeOf(obj));
Doing this returns the wrong size because obj.p returns a size of 4 and not 10. Because of the memory taken by the pointer pointing to the byte array is 4 bytes.
Is there a better way?
The return value is correct, p is a pointer, it takes 4 bytes.
You cannot leave it this way, there are two memory allocations. The marshaller allocated the memory for the array. It created a SAFEARRAY, a COM array type. Pretty unlikely that your C code is going to be happy with that. Declare it like this instead:
[StructLayout(LayoutKind.Sequential)]
public class OUR_MEM_STR {
public IntPtr p;
public int len;
};
And use Marshal.AllocHGlobal(10) to assign p. Don't forget to clean-up again.
Don't pass true to StructureToPtr(), the memory allocated by AllocHGlobal() isn't initialized. That's going to randomly crash your program. You must pass false.
Lets say I have a struct with more than hundred elements with complex names. And I am passing a struct of the struct type described to a function using ref, like this:
void Foo(ref mystruct a)
{
"I want to modify members or fields of struct a, like this:
a[0] = 10;
a[100] = 11;"
}
Thanks!
Maybe you should re-examine your choice of data structure. Perhaps a dictionary would be better suited?
It's a strange request as you're expecting the order of the fields to be significant, but I suspect you could do this through Reflection or the TypeDescriptor.
I would revise my code sample below to use proper property names with constants, but if you know the property names, just call the properties directly and save yourself from the reflection overhead. Otherwise, use a dictionary with constants.
/* yeah, probably not a good idea.
public void Foo(ref MyStruct a)
{
TypeDescriptor.GetProperties(a)[0].SetValue(a, 10);
}
*/
While you can use the struct LayoutKind attribute to force simple types to share memory like a "C" Union, you still cannot make an array share memory with simple types because ref types (aka garbage collected types) don't work with the attribute. The concept of C shortcuts like memset of a struct don't map to C# in any way, because C# is a safe language. In fact, that is a Good Thing. Many bugs have come from these kinds of memory addressing shortcuts.
If you want to simulate this behavior, create a class with properties that map to specific members of a backing array, but again, why do this? There are much better data structures to suit your needs in C# such as List, SortedList, Dictionary, Map, Stack, etc. that are safe.
You can do this in .NET BUT as several others have already posted: DO NOT DO IT.
Some Code
a.GetType().GetProperties() [0].SetValue (a, newvalue, null);
EDIT:
several reasons not to do this:
the order is not guaranteed !
what happens when there are no properties ?
what happens with readonly properties ?
So again: DO NOT DO THIS!
I will probably burn in hell, but...
Obviously horrible and not recommended and only works if your fields are all ints with default layout...
internal class Program
{
private struct MyStruct
{
//Must be all Int32
public int Field1, Field2, Field3;
}
private static void Main()
{
MyStruct str = new MyStruct {Field1 = 666, Field2 = 667, Field3 = 668};
int[] array = ToArray(str);
array[0] = 100;
array[1] = 200;
array[2] = 300;
str = FromArray(array);
}
private static int[] ToArray(MyStruct str)
{
IntPtr ptr = IntPtr.Zero;
try
{
int size = GetInt32ArraySize(str);
int[] array = new int[size];
ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, array, 0, size);
return array;
}
finally
{
Marshal.FreeHGlobal(ptr);
}
}
private static MyStruct FromArray(int[] arr)
{
IntPtr ptr = IntPtr.Zero;
try
{
MyStruct str = new MyStruct();
int size = GetInt32ArraySize(str);
ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(arr, 0, ptr, size);
str = (MyStruct)Marshal.PtrToStructure(ptr, str.GetType());
return str;
}
finally
{
Marshal.FreeHGlobal(ptr);
}
}
private static int GetInt32ArraySize(MyStruct str)
{
return Marshal.SizeOf(str) / Marshal.SizeOf(typeof(Int32));
}
}
How do I marshal this C++ type?
The ABS_DATA structure is used to associate an arbitrarily long data block with the length information. The declared length of the Data array is 1, but the actual length is given by the Length member.
typedef struct abs_data {
ABS_DWORD Length;
ABS_BYTE Data[ABS_VARLEN];
} ABS_DATA;
I tried the following code, but it's not working. The data variable is always empty and I'm sure it has data in there.
[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential, CharSet = System.Runtime.InteropServices.CharSet.Ansi)]
public struct abs_data
{
/// ABS_DWORD->unsigned int
public uint Length;
/// ABS_BYTE[1]
[System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst = 1)]
public string Data;
}
Old question, but I recently had to do this myself and all the existing answers are poor, so...
The best solution for marshaling a variable-length array in a struct is to use a custom marshaler. This lets you control the code that the runtime uses to convert between managed and unmanaged data. Unfortunately, custom marshaling is poorly-documented and has a few bizarre limitations. I'll cover those quickly, then go over the solution.
Annoyingly, you can't use custom marshaling on an array element of a struct or class. There's no documented or logical reason for this limitation, and the compiler won't complain, but you'll get an exception at runtime. Also, there's a function that custom marshalers must implement, int GetNativeDataSize(), which is obviously impossible to implement accurately (it doesn't pass you an instance of the object to ask its size, so you can only go off the type, which is of course variable size!) Fortunately, this function doesn't matter. I've never seen it get called, and it the custom marshaler works fine even if it returns a bogus value (one MSDN example has it return -1).
First of all, here's what I think your native prototype might look like (I'm using P/Invoke here, but it works for COM too):
// Unmanaged C/C++ code prototype (guess)
//void DoThing (ABS_DATA *pData);
// Guess at your managed call with the "marshal one-byte ByValArray" version
//[DllImport("libname.dll")] public extern void DoThing (ref abs_data pData);
Here's the naïve version of how you might have used a custom marshaler (which really ought to have worked). I'll get to the marshaler itself in a bit...
[StructLayout(LayoutKind.Sequential)]
public struct abs_data
{
// Don't need the length as a separate filed; managed arrays know it.
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<byte>))]
public byte[] Data;
}
// Now you can just pass the struct but it takes arbitrary sizes!
[DllImport("libname.dll")] public extern void DoThing (ref abs_data pData);
Unfortunately, at runtime, you apparently can't marshal arrays inside data structures as anything except SafeArray or ByValArray. SafeArrays are counted, but they look nothing like the (extremely common) format that you're looking for here. So that won't work. ByValArray, of course, requires that the length be known at compile time, so that doesn't work either (as you ran into). Bizarrely, though, you can use custom marshaling on array parameters, This is annoying because you have to put the MarshalAsAttribute on every parameter that uses this type, instead of just putting it on one field and having that apply everywhere you use the type containing that field, but c'est la vie. It looks like this:
[StructLayout(LayoutKind.Sequential)]
public struct abs_data
{
// Don't need the length as a separate filed; managed arrays know it.
// This isn't an array anymore; we pass an array of this instead.
public byte Data;
}
// Now you pass an arbitrary-sized array of the struct
[DllImport("libname.dll")] public extern void DoThing (
// Have to put this huge stupid attribute on every parameter of this type
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<abs_data>))]
// Don't need to use "ref" anymore; arrays are ref types and pass as pointer-to
abs_data[] pData);
In that example, I preserved the abs_data type, in case you want to do something special with it (constructors, static functions, properties, inheritance, whatever). If your array elements consisted of a complex type, you would modify the struct to represent that complex type. However, in this case, abs_data is basically just a renamed byte - it's not even "wrapping" the byte; as far as the native code is concerned it's more like a typedef - so you can just pass an array of bytes and skip the struct entirely:
// Actually, you can just pass an arbitrary-length byte array!
[DllImport("libname.dll")] public extern void DoThing (
// Have to put this huge stupid attribute on every parameter of this type
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<byte>))]
byte[] pData);
OK, so now you can see how to declare the array element type (if needed), and how to pass the array to an unmanaged function. However, we still need that custom marshaler. You should read "Implementing the ICustomMarshaler Interface" but I'll cover this here, with inline comments. Note that I use some shorthand conventions (like Marshal.SizeOf<T>()) that require .NET 4.5.1 or higher.
// The class that does the marshaling. Making it generic is not required, but
// will make it easier to use the same custom marshaler for multiple array types.
public class ArrayMarshaler<T> : ICustomMarshaler
{
// All custom marshalers require a static factory method with this signature.
public static ICustomMarshaler GetInstance (String cookie)
{
return new ArrayMarshaler<T>();
}
// This is the function that builds the managed type - in this case, the managed
// array - from a pointer. You can just return null here if only sending the
// array as an in-parameter.
public Object MarshalNativeToManaged (IntPtr pNativeData)
{
// First, sanity check...
if (IntPtr.Zero == pNativeData) return null;
// Start by reading the size of the array ("Length" from your ABS_DATA struct)
int length = Marshal.ReadInt32(pNativeData);
// Create the managed array that will be returned
T[] array = new T[length];
// For efficiency, only compute the element size once
int elSiz = Marshal.SizeOf<T>();
// Populate the array
for (int i = 0; i < length; i++)
{
array[i] = Marshal.PtrToStructure<T>(pNativeData + sizeof(int) + (elSiz * i));
}
// Alternate method, for arrays of primitive types only:
// Marshal.Copy(pNativeData + sizeof(int), array, 0, length);
return array;
}
// This is the function that marshals your managed array to unmanaged memory.
// If you only ever marshal the array out, not in, you can return IntPtr.Zero
public IntPtr MarshalManagedToNative (Object ManagedObject)
{
if (null == ManagedObject) return IntPtr.Zero;
T[] array = (T[])ManagedObj;
int elSiz = Marshal.SizeOf<T>();
// Get the total size of unmanaged memory that is needed (length + elements)
int size = sizeof(int) + (elSiz * array.Length);
// Allocate unmanaged space. For COM, use Marshal.AllocCoTaskMem instead.
IntPtr ptr = Marshal.AllocHGlobal(size);
// Write the "Length" field first
Marshal.WriteInt32(ptr, array.Length);
// Write the array data
for (int i = 0; i < array.Length; i++)
{ // Newly-allocated space has no existing object, so the last param is false
Marshal.StructureToPtr<T>(array[i], ptr + sizeof(int) + (elSiz * i), false);
}
// If you're only using arrays of primitive types, you could use this instead:
//Marshal.Copy(array, 0, ptr + sizeof(int), array.Length);
return ptr;
}
// This function is called after completing the call that required marshaling to
// unmanaged memory. You should use it to free any unmanaged memory you allocated.
// If you never consume unmanaged memory or other resources, do nothing here.
public void CleanUpNativeData (IntPtr pNativeData)
{
// Free the unmanaged memory. Use Marshal.FreeCoTaskMem if using COM.
Marshal.FreeHGlobal(pNativeData);
}
// If, after marshaling from unmanaged to managed, you have anything that needs
// to be taken care of when you're done with the object, put it here. Garbage
// collection will free the managed object, so I've left this function empty.
public void CleanUpManagedData (Object ManagedObj)
{ }
// This function is a lie. It looks like it should be impossible to get the right
// value - the whole problem is that the size of each array is variable!
// - but in practice the runtime doesn't rely on this and may not even call it.
// The MSDN example returns -1; I'll try to be a little more realistic.
public int GetNativeDataSize ()
{
return sizeof(int) + Marshal.SizeOf<T>();
}
}
Whew, that was long! Well, there you have it. I hope people see this, because there's a lot of bad answers and misunderstanding out there...
It is not possible to marshal structs containing variable-length arrays (but it is possible to marshal variable-length arrays as function parameters). You will have to read your data manually:
IntPtr nativeData = ... ;
var length = Marshal.ReadUInt32 (nativeData) ;
var bytes = new byte[length] ;
Marshal.Copy (new IntPtr ((long)nativeData + 4), bytes, 0, length) ;
If the data being saved isn't a string, you don't have to store it in a string. I usually do not marshal to a string unless the original data type was a char*. Otherwise a byte[] should do.
Try:
[MarshalAs(UnmanagedType.ByValArray, SizeConst=[whatever your size is]]
byte[] Data;
If you need to convert this to a string later, use:
System.Text.Encoding.UTF8.GetString(your byte array here).
Obviously, you need to vary the encoding to what you need, though UTF-8 usually is sufficient.
I see the problem now, you have to marshal a VARIABLE length array. The MarshalAs does not allow this and the array will have to be sent by reference.
If the array length is variable, your byte[] needs to be an IntPtr, so you would use,
IntPtr Data;
Instead of
[MarshalAs(UnmanagedType.ByValArray, SizeConst=[whatever your size is]]
byte[] Data;
You can then use the Marshal class to access the underlying data.
Something like:
uint length = yourABSObject.Length;
byte[] buffer = new byte[length];
Marshal.Copy(buffer, 0, yourABSObject.Data, length);
You may need to clean up your memory when you are finished to avoid a leak, though I suspect the GC will clean it up when yourABSObject goes out of scope. Anyway, here is the cleanup code:
Marshal.FreeHGlobal(yourABSObject.Data);
You are trying to marshal something that is a byte[ABS_VARLEN] as if it were a string of length 1. You'll need to figure out what the ABS_VARLEN constant is and marshal the array as:
[MarshalAs(UnmanagedType.LPArray, SizeConst = 1024)]
public byte[] Data;
(The 1024 there is a placeholder; fill in whatever the actual value of ASB_VARLEN is.)
In my opinion, it's simpler and more efficient to pin the array and take its address.
Assuming you need to pass abs_data to myNativeFunction(abs_data*):
public struct abs_data
{
public uint Length;
public IntPtr Data;
}
[DllImport("myDll.dll")]
static extern void myNativeFunction(ref abs_data data);
void CallNativeFunc(byte[] data)
{
GCHandle pin = GCHandle.Alloc(data, GCHandleType.Pinned);
abs_data tmp;
tmp.Length = data.Length;
tmp.Data = pin.AddrOfPinnedObject();
myNativeFunction(ref tmp);
pin.Free();
}