Develop Reference

How can I pass a byte array by reference from c# to c++

I am using a function from a c++ .dll from within C#. The .dll is partially documented, it was created to read out results from a result database. The function has the following signature:
cdb_get(int index, int kwh, int kwl, void *s, int *ls, int nrew);
In many cases I know the structure of the data I am reading, so I can just create a struct with the right format and read into it using regular pInvoke.
However in some cases, I need to know the first bytes of the data to figure out the correct data type. Ideally I would thus like to read the data into a byte array and then convert this further to the struct I need. I'd like to do something like this:
[DllImport("name_of_dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int cdb_get(
int index,
int kwh,
int kwl,
ref byte[] data,
ref int dataLen,
int pos);
But that doesn't seem to work (should it in theory?). Is there a way to make this work with a byte array and passing it by reference?
So far I am working arount it by using an IntPtr and then marshalling by hand. I need to guess the pointer size though, I guess that's unavoidable? Is there a different way to handle the marshalling?
IntPtr data = Marshal.AllocHGlobal(128);
int size = 128;
int returnValue = cdbGet(index, kwh, kwl, data, ref size, 1);
byteArray = new byte[size];
Marshal.Copy(data, byteArray, 0, size);
Marshal.FreeHGlobal(data);
Any help is appreciated, thanks a lot.

I can not post a comment, so I write here.
This seems same to your question, it may help you.

Marshaling struct with dynamic array size (incorrect size) [duplicate]

How do I marshal this C++ type?
The ABS_DATA structure is used to associate an arbitrarily long data block with the length information. The declared length of the Data array is 1, but the actual length is given by the Length member.
typedef struct abs_data {
ABS_DWORD Length;
ABS_BYTE Data[ABS_VARLEN];
} ABS_DATA;
I tried the following code, but it's not working. The data variable is always empty and I'm sure it has data in there.
[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential, CharSet = System.Runtime.InteropServices.CharSet.Ansi)]
public struct abs_data
{
/// ABS_DWORD->unsigned int
public uint Length;
/// ABS_BYTE[1]
[System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst = 1)]
public string Data;
}

Old question, but I recently had to do this myself and all the existing answers are poor, so...
The best solution for marshaling a variable-length array in a struct is to use a custom marshaler. This lets you control the code that the runtime uses to convert between managed and unmanaged data. Unfortunately, custom marshaling is poorly-documented and has a few bizarre limitations. I'll cover those quickly, then go over the solution.
Annoyingly, you can't use custom marshaling on an array element of a struct or class. There's no documented or logical reason for this limitation, and the compiler won't complain, but you'll get an exception at runtime. Also, there's a function that custom marshalers must implement, int GetNativeDataSize(), which is obviously impossible to implement accurately (it doesn't pass you an instance of the object to ask its size, so you can only go off the type, which is of course variable size!) Fortunately, this function doesn't matter. I've never seen it get called, and it the custom marshaler works fine even if it returns a bogus value (one MSDN example has it return -1).
First of all, here's what I think your native prototype might look like (I'm using P/Invoke here, but it works for COM too):
// Unmanaged C/C++ code prototype (guess)
//void DoThing (ABS_DATA *pData);
// Guess at your managed call with the "marshal one-byte ByValArray" version
//[DllImport("libname.dll")] public extern void DoThing (ref abs_data pData);
Here's the naïve version of how you might have used a custom marshaler (which really ought to have worked). I'll get to the marshaler itself in a bit...
[StructLayout(LayoutKind.Sequential)]
public struct abs_data
{
// Don't need the length as a separate filed; managed arrays know it.
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<byte>))]
public byte[] Data;
}
// Now you can just pass the struct but it takes arbitrary sizes!
[DllImport("libname.dll")] public extern void DoThing (ref abs_data pData);
Unfortunately, at runtime, you apparently can't marshal arrays inside data structures as anything except SafeArray or ByValArray. SafeArrays are counted, but they look nothing like the (extremely common) format that you're looking for here. So that won't work. ByValArray, of course, requires that the length be known at compile time, so that doesn't work either (as you ran into). Bizarrely, though, you can use custom marshaling on array parameters, This is annoying because you have to put the MarshalAsAttribute on every parameter that uses this type, instead of just putting it on one field and having that apply everywhere you use the type containing that field, but c'est la vie. It looks like this:
[StructLayout(LayoutKind.Sequential)]
public struct abs_data
{
// Don't need the length as a separate filed; managed arrays know it.
// This isn't an array anymore; we pass an array of this instead.
public byte Data;
}
// Now you pass an arbitrary-sized array of the struct
[DllImport("libname.dll")] public extern void DoThing (
// Have to put this huge stupid attribute on every parameter of this type
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<abs_data>))]
// Don't need to use "ref" anymore; arrays are ref types and pass as pointer-to
abs_data[] pData);
In that example, I preserved the abs_data type, in case you want to do something special with it (constructors, static functions, properties, inheritance, whatever). If your array elements consisted of a complex type, you would modify the struct to represent that complex type. However, in this case, abs_data is basically just a renamed byte - it's not even "wrapping" the byte; as far as the native code is concerned it's more like a typedef - so you can just pass an array of bytes and skip the struct entirely:
// Actually, you can just pass an arbitrary-length byte array!
[DllImport("libname.dll")] public extern void DoThing (
// Have to put this huge stupid attribute on every parameter of this type
[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(ArrayMarshaler<byte>))]
byte[] pData);
OK, so now you can see how to declare the array element type (if needed), and how to pass the array to an unmanaged function. However, we still need that custom marshaler. You should read "Implementing the ICustomMarshaler Interface" but I'll cover this here, with inline comments. Note that I use some shorthand conventions (like Marshal.SizeOf<T>()) that require .NET 4.5.1 or higher.
// The class that does the marshaling. Making it generic is not required, but
// will make it easier to use the same custom marshaler for multiple array types.
public class ArrayMarshaler<T> : ICustomMarshaler
{
// All custom marshalers require a static factory method with this signature.
public static ICustomMarshaler GetInstance (String cookie)
{
return new ArrayMarshaler<T>();
}
// This is the function that builds the managed type - in this case, the managed
// array - from a pointer. You can just return null here if only sending the
// array as an in-parameter.
public Object MarshalNativeToManaged (IntPtr pNativeData)
{
// First, sanity check...
if (IntPtr.Zero == pNativeData) return null;
// Start by reading the size of the array ("Length" from your ABS_DATA struct)
int length = Marshal.ReadInt32(pNativeData);
// Create the managed array that will be returned
T[] array = new T[length];
// For efficiency, only compute the element size once
int elSiz = Marshal.SizeOf<T>();
// Populate the array
for (int i = 0; i < length; i++)
{
array[i] = Marshal.PtrToStructure<T>(pNativeData + sizeof(int) + (elSiz * i));
}
// Alternate method, for arrays of primitive types only:
// Marshal.Copy(pNativeData + sizeof(int), array, 0, length);
return array;
}
// This is the function that marshals your managed array to unmanaged memory.
// If you only ever marshal the array out, not in, you can return IntPtr.Zero
public IntPtr MarshalManagedToNative (Object ManagedObject)
{
if (null == ManagedObject) return IntPtr.Zero;
T[] array = (T[])ManagedObj;
int elSiz = Marshal.SizeOf<T>();
// Get the total size of unmanaged memory that is needed (length + elements)
int size = sizeof(int) + (elSiz * array.Length);
// Allocate unmanaged space. For COM, use Marshal.AllocCoTaskMem instead.
IntPtr ptr = Marshal.AllocHGlobal(size);
// Write the "Length" field first
Marshal.WriteInt32(ptr, array.Length);
// Write the array data
for (int i = 0; i < array.Length; i++)
{ // Newly-allocated space has no existing object, so the last param is false
Marshal.StructureToPtr<T>(array[i], ptr + sizeof(int) + (elSiz * i), false);
}
// If you're only using arrays of primitive types, you could use this instead:
//Marshal.Copy(array, 0, ptr + sizeof(int), array.Length);
return ptr;
}
// This function is called after completing the call that required marshaling to
// unmanaged memory. You should use it to free any unmanaged memory you allocated.
// If you never consume unmanaged memory or other resources, do nothing here.
public void CleanUpNativeData (IntPtr pNativeData)
{
// Free the unmanaged memory. Use Marshal.FreeCoTaskMem if using COM.
Marshal.FreeHGlobal(pNativeData);
}
// If, after marshaling from unmanaged to managed, you have anything that needs
// to be taken care of when you're done with the object, put it here. Garbage
// collection will free the managed object, so I've left this function empty.
public void CleanUpManagedData (Object ManagedObj)
{ }
// This function is a lie. It looks like it should be impossible to get the right
// value - the whole problem is that the size of each array is variable!
// - but in practice the runtime doesn't rely on this and may not even call it.
// The MSDN example returns -1; I'll try to be a little more realistic.
public int GetNativeDataSize ()
{
return sizeof(int) + Marshal.SizeOf<T>();
}
}
Whew, that was long! Well, there you have it. I hope people see this, because there's a lot of bad answers and misunderstanding out there...

It is not possible to marshal structs containing variable-length arrays (but it is possible to marshal variable-length arrays as function parameters). You will have to read your data manually:
IntPtr nativeData = ... ;
var length = Marshal.ReadUInt32 (nativeData) ;
var bytes = new byte[length] ;
Marshal.Copy (new IntPtr ((long)nativeData + 4), bytes, 0, length) ;

If the data being saved isn't a string, you don't have to store it in a string. I usually do not marshal to a string unless the original data type was a char*. Otherwise a byte[] should do.
Try:
[MarshalAs(UnmanagedType.ByValArray, SizeConst=[whatever your size is]]
byte[] Data;
If you need to convert this to a string later, use:
System.Text.Encoding.UTF8.GetString(your byte array here).
Obviously, you need to vary the encoding to what you need, though UTF-8 usually is sufficient.
I see the problem now, you have to marshal a VARIABLE length array. The MarshalAs does not allow this and the array will have to be sent by reference.
If the array length is variable, your byte[] needs to be an IntPtr, so you would use,
IntPtr Data;
Instead of
[MarshalAs(UnmanagedType.ByValArray, SizeConst=[whatever your size is]]
byte[] Data;
You can then use the Marshal class to access the underlying data.
Something like:
uint length = yourABSObject.Length;
byte[] buffer = new byte[length];
Marshal.Copy(buffer, 0, yourABSObject.Data, length);
You may need to clean up your memory when you are finished to avoid a leak, though I suspect the GC will clean it up when yourABSObject goes out of scope. Anyway, here is the cleanup code:
Marshal.FreeHGlobal(yourABSObject.Data);

You are trying to marshal something that is a byte[ABS_VARLEN] as if it were a string of length 1. You'll need to figure out what the ABS_VARLEN constant is and marshal the array as:
[MarshalAs(UnmanagedType.LPArray, SizeConst = 1024)]
public byte[] Data;
(The 1024 there is a placeholder; fill in whatever the actual value of ASB_VARLEN is.)

In my opinion, it's simpler and more efficient to pin the array and take its address.
Assuming you need to pass abs_data to myNativeFunction(abs_data*):
public struct abs_data
{
public uint Length;
public IntPtr Data;
}
[DllImport("myDll.dll")]
static extern void myNativeFunction(ref abs_data data);
void CallNativeFunc(byte[] data)
{
GCHandle pin = GCHandle.Alloc(data, GCHandleType.Pinned);
abs_data tmp;
tmp.Length = data.Length;
tmp.Data = pin.AddrOfPinnedObject();
myNativeFunction(ref tmp);
pin.Free();
}

Return string from unmanaged dll to C#

I'm sorry to ask this here since I'm sure it must be answered "out there", but I've been stuck on this for several months now, and none of the solutions I've found have worked for me.
I have the following VB code that works:
Declare Function DeviceSendRead Lib "unmanaged.dll" (ByVal sCommand As String, ByVal sReply As String, ByVal sError As String, ByVal Timeout As Double) As Integer
Dim err As Integer
Dim outstr As String
Dim readstr As String
Dim errstr As String
outstr = txtSend.Text
readstr = Space(4000)
errstr = Space(100)
Timeout = 10
err = DeviceSendRead(outstr, readstr, errstr, Timeout)
and I am trying to implement it in a C# project. The best equivalent I have been able to find is:
[DllImport("unmanaged.dll")] public static extern int DeviceSendRead(String outstr, StringBuilder readstr, StringBuilder errstr, double Timeout);
int err;
StringBuilder readstr = new StringBuilder(4000);
StringBuilder errstr = new StringBuilder(100);
err = DeviceSendRead(txtSend.Text, readstr, errstr, 10);
However, when I run this, the application freezes and I must force quit it. By experimenting with ref and out, I have occasionally managed to make it crash rather than freeze, but the only "progress" I have achieved is to replace the dll function call with:
DeviceSendRead(txtSend.Text, null, null, 10);
This prevents the crash, but of course does nothing (that I can detect). I'm therefore assuming that it's the manner of passing the two return string parameters that is causing the problem. If anyone can suggest what I might be doing wrong, I'd be very happy to hear it. Thanks.

I have reached an answer, which I will record here for completeness, with grateful thanks to all those who pointed me in the right direction.
According to this post elsewhere, the use of .NET Reflector on similar VB code suggests the need to use the string type in place of my StringBuilder, as suggested here by Alex Mendez, JamieSee and Austin Salonen, together with explicit marshaling, as suggested by Nanhydrin, but utilising the unmanaged type VBByRefStr rather than AnsiBStr. The final key to the puzzle is that the string parameter then needs to be passed by reference using the ref keyword.
I can confirm that this works, and that my final working C# code is therefore:
[DllImport("unmanaged.dll", CharSet = CharSet.Ansi)]
public static extern short DeviceSendRead(
[MarshalAs(UnmanagedType.VBByRefStr)] ref string sCommand,
[MarshalAs(UnmanagedType.VBByRefStr)] ref string sReply,
[MarshalAs(UnmanagedType.VBByRefStr)] ref string sError,
double Timeout);
short err;
string outstr = txtSend.Text;
string readstr = new string(' ', 4000);
string errstr = new string(' ', 100);
err = DeviceSendRead(ref outstr, ref readstr, ref errstr, 10);
I hope this is useful to others facing a similar issue.

Try this:
[DllImport("unmanaged.dll")]
public static extern int DeviceSendRead(string outString, string readString, string errorString, double timeout);
int err;
string outstr;
string readstr;
string errstr =
outstr = txtSend.Text;
readstr = new string(' ', 4000);
errstr = new string(' ', 100);
double timeout = 10;
err = DeviceSendRead(outstr, readstr, errstr, timeout);

Try this as an equivalent:
string readstr = new string(' ', 4000);
string errstr = new string(' ', 1000);

Default Marshalling for strings
Default Marshalling behaviour
You may need to be more specific in your dllimport declaration and add in some MarshalAs attributes, if you have more details on what type of strings the called function is expecting (Ansi, Unicode, null terminated, etc.) then that would help.
In fact it expecting null terminated strings could perhaps explain why it's hanging rather than erroring out.
[DllImport("unmanaged.dll", EntryPoint="DeviceSendRead")]
public static extern int DeviceSendRead(string outString, [MarshalAs(UnmanagedType.AnsiBStr)]string readString, string errorString, double timeout);
You might also need to explicitly state that your parameters are input, output, or both by using the parameter attributes [In, Out].

[DllImport("unmanaged.dll", EntryPoint="DeviceSendRead")]
public static extern int DeviceSendRead(string outstr, string readstr, string errstr, double Timeout);
You cannot marshal a StringBuilder here. There are some rules to follow for marshalling StringBuilder (see CLR Inside Out: Marshaling between Managed and Unmanaged Code):
StringBuilder and Marshaling
The CLR marshaler has built-in knowledge of the StringBuilder type and
treats it differently from other types. By default, StringBuilder is
passed as [InAttribute, OutAttribute]. StringBuilder is special
because it has a Capacity property that can determine the size of the
required buffer at run time, and it can be changed dynamically.
Therefore, during the marshaling process, the CLR can pin
StringBuilder, directly pass the address of internal buffer used in
StringBuilder, and allow the contents of this buffer to be changed by
native code in place.
To take full advantage of StringBuilder, you'll need to follow all of
these rules:
1.Don't pass StringBuilder by reference (using out or ref). Otherwise, the CLR will expect the signature of this argument to be wchar_t **
instead of wchar_t *, and it won't be able to pin StringBuilder's
internal buffer. Performance will be significantly degraded.
2.Use StringBuilder when the unmanaged code is using Unicode. Otherwise, the CLR will have to make a copy of the string and convert
it between Unicode and ANSI, thus degrading performance. Usually you
should marshal StringBuilder as LPARRAY of Unicode characters or as
LPWSTR.
3.Always specify the capacity of StringBuilder in advance and make sure the capacity is big enough to hold the buffer. The best practice
on the unmanaged code side is to accept the size of the string buffer
as an argument to avoid buffer overruns. In COM, you can also use
size_is in IDL to specify the size.
Rule 3 doesn't seem like it is satisied here.

Passing pointers from unmanaged code

I have a C# project that imports a C dll, the dll has this function:
int primary_read_serial(int handle, int *return_code, int *serial, int length);
I want to get access to the serial parameter. I've actually got it to return one character of the serial parameter, but I'm not really sure what I'm doing and would like to understand what is going, and of course get it working.
So, I'm very sure that the dll is being accessed, other functions without pointers work fine. How do I handle pointers? Do I have to marshal it? Maybe I have to have a fixed place to put the data it?
An explanation would be great.
Thanks!
Richard

You will have to use an IntPtr and Marshal that IntPtr into whatever C# structure you want to put it in. In your case you will have to marshal it to an int[].
This is done in several steps:
Allocate an unmanaged handle
Call the function with the unamanged array
Convert the array to managed byte array
Convert byte array to int array
Release unmanaged handle
That code should give you an idea:
// The import declaration
[DllImport("Library.dll")]
public static extern int primary_read_serial(int, ref int, IntPtr, int) ;
// Allocate unmanaged buffer
IntPtr serial_ptr = Marshal.AllocHGlobal(length * sizeof(int));
try
{
// Call unmanaged function
int return_code;
int result = primary_read_serial(handle, ref return_code, serial_ptr, length);
// Safely marshal unmanaged buffer to byte[]
byte[] bytes = new byte[length * sizeof(int)];
Marshal.Copy(serial_ptr, bytes, 0, length);
// Converter to int[]
int[] ints = new int[length];
for (int i = 0; i < length; i++)
{
ints[i] = BitConverter.ToInt32(bytes, i * sizeof(int));
}
}
finally
{
Marshal.FreeHGlobal(serial_ptr);
}
Don't forget the try-finally, or you will risk leaking the unmanaged handle.

If I understand what you're trying to do, this should work for you.
unsafe
{
int length = 1024; // Length of data to read.
fixed (byte* data = new byte[length]) // Pins array to stack so a pointer can be retrieved.
{
int returnCode;
int retValue = primary_read_serial(0, &returnCode, data, length);
// At this point, `data` should contain all the read data.
}
}
JaredPar gives you the easy way to do it however, which is just to change your external function declaration so that C# does the marshalling for you in the background.
Hopefully this gives you an idea of what's happening at a slightly lower level anyway.

When writing your P/invoke declaration of that function, try using keyword ref for the pointer parameters like this:
[DllImport("YouDll.dll", EntryPoint = "primary_read_serial")]
public static extern int primary_read_serial(int, ref int, ref int, int) ;
I'm not sure if you need to specify the parameters' name in C#. And remember, when calling that method, you will also have to use ref in the arguments you're passing by reference.

How to marshal unmanaged buffer of packed structs in c#

I am (successfully) calling the Windows FilterSendMessage function in c# using the following pinvoke signature:
[DllImport("fltlib.dll")]
public static extern IntPtr FilterSendMessage(
IntPtr hPort,
IntPtr inBuffer,
UInt32 inBufferSize,
IntPtr outBuffer,
UInt32 outBufferSize,
out UInt32 bytesReturned);
The outBuffer parameter is populated with an arbitrary number of structs (packed one after the other), defined in C as:
typedef struct _BAH_RECORD {
int evt
int len;
WCHAR name[1];
} BAH_RECORD, *PBAH_RECORD;
The name field is assigned a variable length, null-terminated unicode string. The len field describes the total size of the struct in bytes (including the name string). I am confident there's nothing wrong with how the structs are being handled in the unmanaged side of things.
My problem arises when I try and marshal the outBuffer to an instance of the BAH_RECORD struct, defined in c# as:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
public struct BAH_RECORD
{
public UInt32 evt;
public UInt32 len;
public string name;
}
IntPtr outBuffer = Marshal.AllocHGlobal(OUT_BUFFER_SIZE);
hResult = Win32.FilterSendMessage(hPortHandle, inBuffer, IN_BUFFER_SIZE, outBuffer, OUT_BUFFER_SIZE, out bytesReturned);
BAH_RECORD bah = (BAH_RECORD)Marshal.PtrToStructure(outBuffer, typeof(BAH_RECORD));
<snip>
If I try and print/view/display bah.name, I get garbage...
To confirm that outBuffer does actually contain valid data, I did some crude pointer hackery in c# to step though it, calling Marshal.ReadInt32 twice (to cover the first 2 struct fields), and then Marshal.ReadByte a few times to populate a byte[] which I then use as an argument to Encoding.Unicode.GetString()...the string comes out fine, so it's definitely in there, I just can't seem to get the marshaller to handle it correctly (if it even can?)
Any help appreciated
Steve

The problem is that the 'name' string in your C# BAH_RECORD struct is marshaled as a pointer to a string (WCHAR*) but on the C side it is an inline WCHAR buffer. So when you marshal your struct the runtime reads the first four bytes of the buffer as a pointer and then attempts to read the string that it points to.
Unfortunately, there is no way for the runtime to automatically marshal variable sized buffers inside structs so you will need to use manual marshaling (or as you say "pointer hackery"). But when you advance the pointer to point at the buffer you don't need to read in the bytes individually and then convert them to a string - just call Marshal.PtrToStringUni.