Assume that there is a native function with a pure-C interface like the following one, exported from a native DLL:
// NativeDll.cpp
extern "C" void __stdcall FillArray(
int fillValue,
int count,
int* data)
{
// Assume parameters are OK...
// Fill the array
for (int i = 0; i < count; i++)
{
data[i] = fillValue;
}
}
The following P/Invoke works fine (tested with VS2010 SP1):
[DllImport("NativeDll.dll", CallingConvention=CallingConvention.StdCall)]
public static extern void FillArray(
int fillValue,
int count,
[In, Out] int[] data
);
as well as this P/Invoke, same as above, but without the [In, Out] attributes:
[DllImport("NativeDll.dll", CallingConvention=CallingConvention.StdCall)]
public static extern void FillArray(
int fillValue,
int count,
int[] data
);
So, are those [In, Out] attributes optional for marshaling arrays?
What is their purpose, if any?
Is it OK to omit them in our P/Invoke declarations?
No, they are not exactly optional. It just happens to work by accident. It is however a very common accident. It works because the array doesn't actually gets marshaled. The pinvoke marshaller sees that the C# array is already compatible with the native array so skips the step to create a copy of it. It merely pins the array and passes the pointer to the native code.
This is of course very efficient and you will inevitably get the results back because the native code is directly writing the array elements. So neither the [In] nor the [Out] attributes matter.
It gets much murkier if the array element type isn't that simple. It isn't that easy to identify an element type that's a struct or class type that isn't blittable or whose layout doesn't match after marshaling so the pinvoke marshaller has to make a copy of the array. Especially the layout incompatibility can be very hard to identify because the managed layout is undiscoverable. And can change depending on the jitter that is used. It may work in x86 but not x64 for example, pretty nasty when AnyCPU is selected. Getting it to copy the modified copy back to the C# array does require [Out].
Not sure what to advice, other than that nobody ever got fired for being explicit in their declarations. Perhaps you should always be explicit when the array element type isn't simple so you'll never have an accident.
Related
I'm reading all about structure marshalling between C and C# in 64bit environment without success.
What I need to do is to pass the structure
typedef struct DescStructTa
{
char* pszNa;
unsigned long ulTId;
char* pszT;
unsigned short usRId;
unsigned long ulOff;
unsigned long ulSi;
char szAcc[2];
unsigned char bySSize;
} DescStruct;
from a C# application to a C DLL not made by us calling the method
MT_DLL_DECL long GetAll(UINTPTR ulHandler, DescStruct **ppList, unsigned long *pulNumS);
After long search I wrote the following code:
[StructLayout(LayoutKind.Sequential, Pack = 8)]
public struct PlcSymbolDescStruct
{
[MarshalAs(UnmanagedType.LPStr)]
string pszNa;
UInt32 ulTId;
[MarshalAs(UnmanagedType.LPStr)]
string pszT;
[MarshalAs(UnmanagedType.U2)]
ushort usRId;
UInt32 ulOff;
UInt32 ulSi;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 2)]
string szAcc;
[MarshalAs(UnmanagedType.U1)]
byte bySSize;
}
[DllImport("DataSource.dll", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
[return: MarshalAs(UnmanagedType.U8)]public static extern long GetAll(ulong ulHandler, out DescStruct [] ppList,
out uint pulNumS);
Unfortunately this is still giving me a heap corruption.
If I try to pack less than 8 it gives (as expected) an access violation, but I expected this to work and I'm not able to figure out what could be wrong.
Any suggestion?
Thanks in advance!
I’m not sure you need Pack = 8. The default packing appears to be more or less compatible between C# and C++.
The szAcc field probably uses Unicode. To switch, specify CharSet=CharSet.Ansi in your [StructLayout]
Finally, if your C API has that double pointer to the array of them, you have to allocate that memory in a special way, either CoTaskMemAlloc or LocalAlloc, forgot which one.
It’s easier, also more efficient, to implement 2 functions instead. One to obtain the required size of the array. Another one to write them into caller-allocated buffer, a single pointer in C++ as opposed to double pointer, and UnmanagedType.LPArray in C#.
Update
allocation should not be necessary since the documentation states that the call returns a pointer to it's internal data structure
The runtime doesn’t know what was written in that documentation, attempts to free anyway, and crashes. You can use out IntPtr in C# API, and marshal manually.
If you have modern .NET, use unsafe, cast the IntPtr to PlcSymbolDescStruct* raw pointer, and construct ReadOnlySpan<PlcSymbolDescStruct> from that raw pointer.
if you’re stuck with old desktop .NET, call Marshal.PtrToStructure<PlcSymbolDescStruct> in a loop, incrementing the pointer by Marshal.SizeOf<PlcSymbolDescStruct>() bytes between loop iterations. No need to use unsafe code in this case.
If I have a native code that expects a structure with two fields:
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern int my_method(ref MyStruct myStruct);
// this is what native code expects
public struct MyStruct {
IntPtr First;
IntPtr Second;
}
but instead I pass another struct to it, will it work or not - by design or by accident?
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern int my_method(ref MyLongerStruct myLongerStruct);
// this is what I want to pass to it
public struct MyLongerStruct {
IntPtr First;
IntPtr Second;
object ObjectPointer;
}
Will the object reference added to the end of the struct at C# side somehow affect P/Invoke call?
I shouldn't work. And even more, you need to add and properly set StructLayoutAttribute to the structure, as it explained here
I think, the result should be like this:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct {
IntPtr First;
IntPtr Second;
}
If the total difference in structure is fields added to the end, and you use StructLayout to prevent the compiler from optimizing the memory layout of your struct (as Alex Butenko suggests), then it's unlikely that there will be any negative side effects apart from a slight speed hit.
When you pass a managed struct to an external function via P/Invoke (using the DllImport attribute) there is a marshaling phase that converts your struct to a compatible format for the target. For ref and out parameters the temporary is converted back when the invoked function returns, copying the values back to your struct instance. All of this is abstracted away, although exactly how the marshaling is performed for each member can be tweaked with the right attributes.
This is how the .NET framework handles strings in P/Invoke. Since it can't just send a string instance pointer to an API function that is expecting a char * (the two are nothing alike) there has to be some translation.
The fun part is that the marshaling code doesn't know anything about what the target is expecting other than what you tell it at the C# end, so if you are sending an extended version of the structure it will do the whole thing. At the other end the native code will get a pointer to a memory block containing the information it's expecting, and it won't have any way to tell that there is more after the end of the structure.
Apart from that, no problem... as long as you're passing by reference and not by value. Passing structs by value is something that should raise big red stop signs all over your brain. Don't do it, it's evil.
I have a C++ dll which is used to card printing( ID cards ). My implementation done using C#.Net. I used following code to call c++ dll.
[DllImport(#"J230i.dll",CallingConvention = CallingConvention.Cdecl,SetLastError=true)]
public static extern int N_PrintJobStatus(ref int[] nPrtintjobStatus);
int[] pJob = {0,0,0,0,0,0,0,0} ;
ret = N_PrintJobStatus( ref pJob);
N_PrintJobStatus method signature given as bellow
N_PrintJobStatus(int *pJobStatus )
After calling the method it gives following error
A call to PInvoke function '********!*********.frmCardPrint::N_PrintJobStatus' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.
How can I fix this issue
thank you .....
Your translation is incorrect. An int array, int* does not map to ref int[]. The latter would be marshalled as int**. You need instead to use int[].
[DllImport(#"J230i.dll", CallingConvention = CallingConvention.Cdecl,
SetLastError = true)]
public static extern int N_PrintJobStatus(int[] nPrtintjobStatus);
Allocate the array before calling the function. Presumably you have some way to determine how long it should be. As it stands this function looks like a buffer overrun waiting to happen. How can the function know how long the array is and so take steps to avoid writing beyond its end?
It's not clear that this is the only problem. We cannot be sure that the return type really is int. Or that the calling convention is cdecl. Or that the function really does call SetLastError.
In the Compact Framework 3.5, I am attempting to call an ActiveX object that has an IDL function signature:
HRESULT MyFunc([out] SAFEARRAY(float) *var)
The Interop generation creates the msil
[out] class [mscorlib]System.Array& marshal( safearray float32)
Which seems reasonable enough, but I keep getting a "NotSupportedException". According to an article entitled "Interop: Common issues and debugging techniques" (I can't post more than one hyperlink, it's the first google result for that phrase), in the first bullet point under the "Marshaling" heading, the compact framework doesn't properly marshal SAFEARRAYs.
I have attempted to get around this problem, by manipulating the answer described in this MSDN forum post (Last entry describes his method): http://social.msdn.microsoft.com/forums/en-US/clr/thread/6641abfc-3a9c-4976-a523-43890b2b79a2/
So, I have created the following definition:
[StructLayout(LayoutKind.Sequential)]
struct SafeArray
{
public ushort dimensions; // Count of dimensions in the SAFEARRAY
public ushort features; // Flags to describe SAFEARRAY usage
public uint elementSize; // Size of an array element
public uint locks; // Number of times locked without unlocking
public IntPtr dataPtr; // Pointer to the array data
public uint elementCount; // Element count for first (only) dimension
public int lowerBound; // Lower bound for first (only) dimension
}
And redefined the IDL for the function signature to:
HRESULT MyFunc([out] long *var)
And then issuing the following code:
IntPtr safeArrayPtr = Marshal.AllocCoTaskMem(Marshal.SizeOf(typeof(SafeArray)));
SafeArray safeArray;
safeArray.dimensions = 1;
safeArray.features = 0;
safeArray.elementSize = (uint)(Marshal.SizeOf(typeof(float)));
safeArray.locks = 0;
safeArray.elementCount = 6;
safeArray.lowerBound = 0;
safeArray.dataPtr = Marshal.AllocCoTaskMem((int)(safeArray.elementCount * safeArray.elementSize));
Marshal.StructureToPtr(safeArray, safeArrayPtr, false);
int iTmp = safeArrayPtr.ToInt32();
MyFunc(out iTmp)
While the code appears to succeed, when I try to read back the data values, using the Marshal.Copy(dataPtr, myFloatArr, false) function, I am getting all 0's for the data, which tells me that the pointer that the ActiveX DLL is getting is probably totally bogus and it's off writing into oblivion.
Any suggestions as to what I may have messed up in these definitions, or suggestions for other ways of approaching this problem?
Thanks In Advance...
Well, I've solved this one.
Hopefully, my answer will help others who encounter the same problem. The problem that I was running into was that the [out] tag in a COM tlb declaration means that whatever I pass in will be overwritten with the object created inside the COM library. A rather complicated version of the classic (and very elementary problem) "Pass By Reference vs Pass By Value"
So, the correct marshaling is to use the definition of SafeArray that I posted above.
Don't touch the IDL signature itself - that's not a very clean way of doing it. Instead, use ildasm on the generated Interop library to modify the il from:
[out] class [mscorlib]System.Array& marshal( safearray float32)
to
[out] native int&
and then reassemble with ilasm which will produce a C# function signature
void MyFunc(out IntPtr var)
The calling code then becomes:
IntPtr ip;
SafeArray safeArray;
float []arrFloats;
MyFunc(out ip);
//Marshal the structure itself
Marshal.StructureToPtr(safeArray, ip, false);
//Marshal the data over to .NET
float []arrFloats = new float[safeArray.elementCount];
Marshal.Copy(safeArray.dataPtr, arrFloats, 0, (int)safeArray.elementCount);
Finally, we need to free the memory (remember, we changed the function signature so we're not giving .NET enough information to actually free the memory on its own.
//Don't forget to free both the structure and the object
Marshal.FreeCoTaskMem(safeArray.dataPtr);
Marshal.FreeCoTaskMem(ip);
I have two unmanaged pointers in the form of IntPtr and want to copy data between them. How can I do this? I know the method Marshal.Copy, but it can only copy between unmanaged and managed.
And the second part: Is copying unmanaged data from C# slower than doing it in unmanaged C/C++ using memcpy?
Edit: I would be especially interested in a platform independet implementation.
You can use the win32 memcpy function via P-Invoke.
[DllImport("msvcrt.dll", SetLastError = false)]
static extern IntPtr memcpy(IntPtr dest, IntPtr src, int count);
Apart from the (slight) overhead calling a win32 function from managed code, the actual copy performance should be the same as C/C++ code that uses the same function.
Don't forget that you can also use an unsafe block (and compiler option) and simply copy the data one byte/int/long at a time:
unsafe
{
// srcPtr and destPtr are IntPtr's pointing to valid memory locations
// size is the number of long (normally 4 bytes) to copy
long* src = (long*)srcPtr;
long* dest = (long*)destPtr;
for (int i = 0; i < size / sizeof(long); i++)
{
dest[i] = src[i];
}
}
This removes the platform dependency, but you need to be very careful with the bounds checking and pointer arithmetic.
Try System.Buffer.MemoryCopy, see the bottom of the page for supported target frameworks.
I believe that the main difference between this and the other solutions that use P/Invoke is that this method avoids the P/Invoke for smaller sizes and just does the copying directly.
Here's the guts of the implementation in .NET Core (latest as of 2020-09-04).
Without making comments on performance, purely because I have not tested it. You can achieve the same performance as unmanaged copy by using either CopyMemory or MoveMemory from Kernel32 via interop.
Here is the declaration for CopyMemory
[DllImport("kernel32.dll")]
static extern void CopyMemory(IntPtr destination, IntPtr source, uint length);
You could look at System.Runtime.CompilerServices.Unsafe.CopyBlock
It seems to allow you to copy bytes from the source address (designated by a void*) to the destination address (designated by a void*).
It also overriden to support ref byte as the source and destination.
[edit]
Disappointingly it appears not to be implemented in Mono
[edit] For those who are interested in this and using Unity, you should instead look to Unity's UnsafeUtility.MemCpy
CopyMemory aka RtlCopyMemory aka memcpy() will be just as fast whether called from C# or C (other than the tiny overhead of PInvoking the method itself).
Something to keep in mind, though, is that CopyMemory should only be used when you're sure that the source and destination ranges do not overlap. If they do overlap, you need to use MoveMemory instead, which will be slower.
Here is a declaration for CopyMeSomeMemory, showing how many different ways you can do the same thing in .Net:
[DllImport("kernel32.dll", EntryPoint = "RtlCopyMemory")]
public static extern void CopyMeSomeMemory(IntPtr Destination,
IntPtr Source, uint Length);
For the record, I think Buffer.BlockCopy in .Net just wraps one of these functions, too.