I have a C API with the signature:
int GetBuffer(char* buffer, int* maxSize)
In C, I will call it this way:
char buffer[4096];
int maxSize = 4096;
GetBuffer(buffer, &maxSize);
maxSize is set to the buffer size, and set with the actual size filled.
I need to call it from C#. How do I do that under "safe mode"?
One option is simply to use C# pointer types - this requires unsafe block (or modifier on method/class), and compiling with /unsafe:
[DllImport(...)]
static extern int GetBuffer(byte* buffer, ref int maxSize);
Buffer can be allocated in several different ways. One would be to use a pinned heap array:
fixed (byte* buffer = new byte[4096])
{
int maxSize = buffer.Length;
GetBuffer(buffer, ref maxSize);
}
Another is to use stackalloc, though this is only feasible for small buffers:
byte* buffer = stackalloc byte[4096];
int maxSize = 4096;
GetBuffer(buffer, ref maxSize);
This particular approach is virtually identical to your C code in terms of performance and allocation patterns.
Another option altogether is to use marshaling for heap arrays, and avoid pointers entirely.
[DllImport(...)]
static extern int GetBuffer([Out] byte[] buffer, ref int maxSize);
byte[] buffer = new byte[4096];
int maxSize = buffer.Length;
GetBuffer(buffer, ref maxSize);
This should work without unsafe code.
extern int GetBuffer(IntPtr buffer, ref int bufSize);
// ...
byte[] buf = new byte[kBufSize];
GCHandle handle = GCHandle.Alloc(buf, GCHandleType.Pinned); // possibly expensive call
IntPtr p = handle.AddrOfPinnedObject();
int size = buf.Length;
int ret = GetBuffer(p, ref size);
handle.Free();
You need to use what is called P\Invoke, and generate a function declaration that to reference the C function in the Dll from C#.
However, you have to be very careful when passing buffers in/out of unmanaged code. The framework will take care of some things for you but you may need to ensure that memory that you pass into the unmanaged call doesn't get moved by the Garbage collector.
[DllImport("Kernel32.dll", SetLastError=true)]
static extern Int32 GetBuffer(byte[] buffer,ref Int32 maxSize);
And to use it:
byte[] myBuf = new myBuf[4096];
Int32 maxSize = myBuf.Length;
GetBuffer(myBuf, ref maxSize);
Having a handle to the pointer doesn't fit the "safe mode" model at all; if the resource isn't managed by the Framework, it is unsafe.
One easy and safe option is to create a simple class that wraps the value, or any value like the following code:
public class Value<T> where T: struct
{
public static implicit operator T(Value<T> val)
{
return val.Value;
}
private T _value;
public Value(T value)
{
_value = value;
}
public Value() : this(default)
{
}
public T Value
{
get
{
return _value;
}
set
{
_value = value;
}
}
public override string ToString()
{
return _value.ToString();
}
}
Passing on instances of this class instead of the value itself works almost like working with pointers.
Related
I am looking for some guidance when it comes to call DeviceIoControl from C#, knowing that its generic aspect of accepting pointer parameters isn't always easy to express in C#.
Here are two examples and explanations laid out below.
Example 1:
This works but is cumbersome, you have a disposable scope but you have to pass the parameters to the function and at the end assign the output buffer value back to the variable.
var toc = new CDROM_TOC(); // non blittable
var code = NativeConstants.IOCTL_CDROM_READ_TOC;
using (var scope = new UnmanagedMemoryScope<CDROM_TOC>(toc))
{
if (!UnsafeNativeMethods.DeviceIoControl(Handle, code, IntPtr.Zero, 0, scope.Memory, scope.Size, out _))
return Array.Empty<ITrack>();
toc = scope.Value; // this is weird
}
Example 1 helper:
internal struct UnmanagedMemoryScope<T> : IDisposable where T : struct
{
private bool IsDisposed { get; set; }
public uint Size { get; }
public IntPtr Memory { get; }
public T Value
{
get => Marshal.PtrToStructure<T>(Memory);
set => Marshal.StructureToPtr(value, Memory, true);
}
public UnmanagedMemoryScope(T value)
{
var size = Marshal.SizeOf<T>();
Memory = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(value, Memory, false);
Size = (uint)size;
IsDisposed = false;
}
public void Dispose()
{
if (IsDisposed)
return;
if (Memory != default)
Marshal.FreeHGlobal(Memory);
IsDisposed = true;
}
}
Example 2:
This one is already much more friendly, wrappers do marshalling and the value passed is ref.
var toc = new CDROM_TOC(); // non blittable
var code = NativeConstants.IOCTL_CDROM_READ_TOC;
var ioctl = DeviceIoControl(Handle, code, ref toc);
// ...
Example 2 helper 1:
private static bool DeviceIoControl<TTarget>(
SafeFileHandle handle, uint code, ref TTarget target)
where TTarget : struct
{
var sizeOf = Marshal.SizeOf<TTarget>();
var intPtr = Marshal.AllocHGlobal(sizeOf);
Marshal.StructureToPtr(target, intPtr, false);
var ioctl = UnsafeNativeMethods.DeviceIoControl(
handle,
code,
IntPtr.Zero,
0u,
intPtr,
(uint)sizeOf,
out var lpBytesReturned
);
target = Marshal.PtrToStructure<TTarget>(intPtr);
Marshal.FreeHGlobal(intPtr);
return ioctl;
}
Example 2 helper 2:
private static bool DeviceIoControl<TTarget, TSource>(
SafeFileHandle handle, uint code, ref TTarget target, ref TSource source)
where TSource : struct
where TTarget : struct
{
var sizeOf1 = Marshal.SizeOf(source);
var sizeOf2 = Marshal.SizeOf(target);
var intPtr1 = Marshal.AllocHGlobal(sizeOf1);
var intPtr2 = Marshal.AllocHGlobal(sizeOf2);
Marshal.StructureToPtr(source, intPtr1, false);
Marshal.StructureToPtr(target, intPtr2, false);
var ioctl = UnsafeNativeMethods.DeviceIoControl(
handle,
code,
intPtr1,
(uint)sizeOf1,
intPtr2,
(uint)sizeOf2,
out var lpBytesReturned
);
Marshal.PtrToStructure(intPtr1, source);
Marshal.PtrToStructure(intPtr2, target);
Marshal.FreeHGlobal(intPtr1);
Marshal.FreeHGlobal(intPtr2);
return ioctl;
}
But I feel that I might be missing something and maybe there's a better approach...
Question:
What are some good tricks when it comes to call DeviceIoControl from C#?
Knowing that,
want to avoid the use of unsafe keyword
there are non-blittable types so fixed is out of question for them
the function accepts arbitrary types, it's only buffers for it in the end
Of course there's the C++/CLI route but well, it's not C# anymore...
Hope that makes sense to you, else let me know.
I usually do it like this.
Parameters structure:
ref struct CDROM_TOC
{
const int MAXIMUM_NUMBER_TRACKS = 100;
public const int sizeInBytes = 4 + MAXIMUM_NUMBER_TRACKS * 8;
readonly Span<byte> buffer;
public CDROM_TOC( Span<byte> buffer )
{
if( buffer.Length != sizeInBytes )
throw new ArgumentException();
this.buffer = buffer;
}
/// <summary>Fixed header of the structure</summary>
public struct Header
{
public ushort length;
public byte firstTrack, lastTrack;
}
/// <summary>Fixed header</summary>
public ref Header header =>
ref MemoryMarshal.Cast<byte, Header>( buffer.Slice( 0, 4 ) )[ 0 ];
public struct TRACK_DATA
{
byte reserved;
public byte controlAndAdr;
public byte trackNumber;
byte reserved2;
public uint address;
}
/// <summary>Tracks collection</summary>
public Span<TRACK_DATA> tracks =>
MemoryMarshal.Cast<byte, TRACK_DATA>( buffer.Slice( 4 ) );
// Make this structure compatible with fixed() statement
public ref byte GetPinnableReference() => ref buffer[ 0 ];
}
Usage example:
CDROM_TOC toc = new CDROM_TOC( stackalloc byte[ CDROM_TOC.sizeInBytes ] );
unsafe
{
fixed( byte* buffer = toc )
{
// Here you have unmanaged pointer for that C interop.
}
}
// If you want to return the tracks, need to copy to managed heap:
var header = toc.header;
return toc.tracks
.Slice( header.firstTrack, header.lastTrack - header.firstTrack + 1 )
.ToArray();
Couple more notes.
The answer assumes you have a modern C#, i.e. .NET 5 or newer, or any version of .NET Core.
The example does use unsafe, but only on the lowest level. If you absolutely don’t want that, use GCHandle instead. With GCHandleType.Pinned, it’s an equivalent to unsafe keyword, only slower.
Unlike your code, this method does not use any heap memory for the interop, neither managed nor native.
The instance of the structure is stack allocated, and it exposes higher-level API to access the fields of that structure. The complete stack is already fixed in memory, the fixed keyword going to nothing for that code, just return the address. Doing nothing is free performance-wise.
My C declarations are as follows:
int myData(uint myHandle, tchar *dataName, long *Time, uint *maxData, DATASTRUCT **data);
typedef struct {
byte Rel;
__int64 Time;
char Validated;
unsigned char Data[1];
} DATASTRUCT;
My C# declarations are as follows:
[DllImport("myData.dll", EntryPoint = "myData")]
public static extern int myData(uint myHandle, [MarshalAs(UnmanagedType.LPTStr)] string dataName, out long Time, out uint maxData, ref DATASTRUCT[] data);
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct DATASTRUCT
{
public sbyte Rel;
public long Time;
public byte Validated;
public double Data;
}
I then call the managed function as follows:
string dataToShow = "description";
long Time;
uint maxData; // How many structs will be returned, i.e., how much data is available?
uint myHandle = 1;
DATASTRUCT[] dataInformation = new DATASTRUCT[3]; // Doesn't it matter what I specify as the array size?
myData(myHandle, dataToShow, out Time, out maxData, ref dataInformation);
Upon execution the above function will return successfully with only one structure even though there are 3 to return. Why is this so?
Additional information; I have tried passing the pointer to a pointer of an array of structs the following ways:
ref DATASTRUCT[] data; // It works, but it only returns one struct
[Out, MarshalAs(UnmanagedType.LPArray)] DATASTRUCT[] data; // returns the number of defined structs with garbage
As I understand it I might need to do some manual marshalling using IntPtr, I do not know how to implement this however, so any advice would be appreciated.
Okay, it seems as though your native library does the allocation, so really all you need to do is provide a pointer through which you can access the allocated data.
Change your API definition to (note, I changed the maxData param to uint, long is 64 bits in .NET and 32 bits in native.
[DllImportAttribute("myData.dll", EntryPoint = "myData")]
public static extern int myData(uint myHandle, [MarshalAs(UnmanagedType.LPTStr)] string dataName, out uint Time, out uint maxData, out IntPtr pData);
Off the top of my head I can't quite remember if you need the out keyword for the final parameter, but I think so.
Then, call myData:
uint nAllocs = 0, time = 0;
IntPtr pAllocs = IntPtr.Zero;
myData(1, "description", out time, out nAllocs, out pAllocs);
Now, pAllocs should point to unmanaged memory, to marshal these into managed memory isn't too difficult:
[StructLayoutAttribute(LayoutKind.Sequential, Pack = 1)]
public struct DATASTRUCT
{
public byte Rel;
public long Time;
public byte Validated;
public IntPtr Data; //pointer to unmanaged string.
}
int szStruct = Marshal.SizeOf(typeof(DATASTRUCT));
DATASTRUCT[] localStructs = new DATASTRUCT[nAllocs];
for(uint i = 0; i < nallocs; i++)
localStructs[i] = (DATASTRUCT)Marshal.PtrToStructure(new IntPtr(pAllocs.ToInt32() + (szStruct * i)), typeof(DATASTRUCT));
And now you should have an array of local structs.
A point to note
You may need to set your project to compile as x86, to standardize the size of an IntPtr to 4 bytes (DWORD) instead of AnyCPU's default 8.
A pointer to a pointer could be represented in your dllimport declaration as ref IntPtr data, so your declaration would become:
[DllImportAttribute("myData.dll", EntryPoint = "myData")]
public static extern int myData(uint myHandle, [MarshalAs(UnmanagedType.LPTStr)] string dataName, out long Time, out uint maxData, ref IntPtr data);
(As an aside, I think a long in C is just equivalent to an int in C#. Long in C# is an Int64, which would be a long long in C)
Marshalling your DATASTRUCT[] to an IntPtr can be done using the GCHandle class
DATASTRUCT [] dataInformation = new DATASTRUCT[3];
GCHandle gch = GCHandle.Alloc(dataInformation , GCHandleType.Pinned);
IntPtr ptr = gch.AddrOfPinnedObject();
myData(myHandle, dataToShow, out Time, out maxData, ref ptr);
//It's absolutely essential you do this next bit so the object can be garbage collected again,
//but it should only be done once the unmanaged code is definitely done with the reference.
gch.Free();
Using the Marshal class and the StructureToPtr or Copy methods of it would also be an option but for the purposes of proving the concept at least the GCHandle should do the trick, it's just not ideal for scenarios where the unmanaged code does long running operations because you've pinned this object in place and the GC can't move it until you free it.
In my code I have c DLL that accepts array of strings:
void Helper::ProcessEvent(PEVENT_RECORD pEvent,wchar_t** OutPutFormattedData)
I'm invoking it with this code:
[DllImport("Helper.dll", EntryPoint = "ProcessEvent")]
internal static extern uint ProcessEvent(
[In, Out]
ref EVENT_RECORD pEvent,
[In, Out]
[MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr)] ref string[] pResult);
In c++ code here is the main code I'm using to fill the array:
for(int i=0;i<list.Size;i++)
{
EventWrapperClass *EWC = new EventWrapperClass();
EWC = list.Events[i];
OutPutFormattedData[i] = new wchar_t [wcslen(HelperFormatMessage(L"%s: %s\n",EWC->GetProperyName(),EWC->GetProperyValue()))+1];
wcscpy(OutPutFormattedData[i] ,HelperFormatMessage(L"%s: %s\n",EWC->GetProperyName(),EWC->GetProperyValue()));
}
And the invoke code:
string[] strArr= new string[1];
NativeHelper.ProcessEvent(ref eventRecord, ref strArr);
I have two questions:
Why when I check the value of the passed array in c# after calling this function I see it's empty (data is exist in c++ code, I debugged it) ?
If I'm allocating memory in c++ dll, where I need to free it? in c++ or c#?
Many thanks!
Edit:
signture of c++:
static __declspec(dllexport) void ProcessEvent(PEVENT_RECORD pEvent, wchar_t** OutPutFormattedData);
If you allocate a (managed or unmanaged) buffer (or array or chars) in C# and then fill in C++:
1. Deallocate in C#.
2. Buffer should be [in] and not have ref or out in C# signature.
If you allocate a buffer (or array of chars) in C++:
1. Pass buffer as [out] out IntPtr.
2. Add deallocate method to C++ signature.
3. Copy buffer into C# buffer or new string
4. Call C++ deallocater from C# with IntPtr.
You can use System.Runtime.InteropServices.Marshal to allocate/deallocate unmanged memory within C#. Do not use to deallocate memory allocated outside .NET!
System.Runtime.InteropServices.Marshal can also be used to copy memory from external buffer (IntPtr) to .NET buffer or string.
Mark imported method as private and wrap with a public (or protected or internal) method that uses the .NET parameters (e.g. that uses System.Runtime.InteropServices.Marshal.Copy and calls external deallocate).
C++:
int ProcessEvent(PEVENT_RECORD eventData, wchar_t*& message)
{
// example code
message = new wchar_t[100];
if (message == 0)
return 1;
}
void DeallocateString(wchar_t* array)
{
delete[] arrayPtr;
}
wchar_t* ErrorCodeToMessage(int errorCode)
{
switch (errorCode)
{
case 0: return 0; // return NULL pointer
case 1: return L"No!!!";
default: return L"WTF!?";
}
}
C#:
[DllImport("Helper.dll", EntryPoint = "ProcessEvent")]
private static extern uint ProcessEventExternal(
[In, Out] ref EventData eventData,
[In, Out, MarshalAs(UnmanagedType.SysInt))] ref IntPtr resultMessages);
[DllImport("Helper.dll", EntryPoint = "DeallocateString")]
private static extern voidDeallocateStringExternal(
[In, MarshalAs(UnmanagedType.SysInt)] IntPtr arrayPtr);
[DllImport("Helper.dll", EntryPoint = "ErrorCodeToMessage")]
private static extern
[return: MarshalAs(UnmanagedType.SysInt)] IntPtr
ErrorCodeToMessageExternal(int errorCode);
public string ProcessEvent(ref EventData eventData)
{
IntPtr resultPtr = IntPtr.Zero;
uint errorCode = ProcessEventExternal(eventData, ref resultPtr);
if (errorCode != null)
{
var errorPtr = ErrorCodeToMessageExternal(errorCode);
// returns constant string - no need to deallocate
var errorMessage = Marshal.PtrToStringUni(errorPtr);
throw new ApplicationException(errorMessage);
}
var result = Marshal.PtrToStringUni(resultPtr);
ExternalDeallocate(resultPtr);
return result;
}
I don't know about question 1, but about question 2:
for(int i=0;i<list.Size;i++)
{
EventWrapperClass *EWC = new EventWrapperClass();
EWC = list.Events[i];
...
}
In the first line, within the loop, you create a new object. In the second line, you assign a different object to your pointer, which is the one stored in list.Events[i].
At this point, nothing points to your first object any longer, and you have a memory leak for every iteration of the loop!
Change the two lines to
EventWrapperClass *EWC = list.Events[i];
as there is no need to create a new object.
Then, if you don't need the event any longer, delete it within the loop and be sure to clear the list afterwards. You may only do this if you are 100% sure the event is not needed any longer. This can easily produce dangling pointers!
I have a c# .net 2.0 CF application that interfaces with a native DLL implementing a function like this:
struct NATIVE_METHOD_REPLY
{
int other_irrelevant_data;
int data_size;
void* data;
}
// reply_buffer will contain an array of NATIVE_METHOD_REPLY structures
// and their data.
//
// returns an error code
int Foo(NATIVE_METHOD_REPLY* reply_buffer, int reply_size);
I've implemented it in C# like this:
[StructLayout(LayoutKind.Sequential)]
internal struct NATIVE_METHOD_REPLY
{
public Int32 OtherIrrelevantData;
public Int16 DataSize;
public IntPtr DataPtr;
}
[DllImport("my_lib.dll", SetLastError = true)]
internal static extern Int32 Foo(byte[] replyBuffer, Int32 replySize);
public byte[] void Bar()
{
// data returned to the user. May be an arbitrary size.
byte[] result_buffer = new byte[256];
// data sent to Foo()
byte[] reply_buffer =
new byte[Marshal.SizeOf(typeof(NativeMethods.NATIVE_METHOD_REPLY)) +
result_buffer.Length];
NativeMethods.Foo(reply_buffer, reply_buffer.Length);
// is there a better way of doing this?
NativeMethods.NATIVE_METHOD_REPLY reply;
GCHandle pinned_reply = GCHandle.Alloc(reply_buffer,
GCHandleType.Pinned);
try
{
reply = (NativeMethods.NATIVE_METHOD_REPLY)Marshal.PtrToStructure(
pinned_reply.AddrOfPinnedObject(),
typeof(NativeMethods.NATIVE_METHOD_REPLY));
Marshal.Copy(reply.DataPtr, result_buffer, 0, reply.DataSize);
}
finally
{
pinned_reply.Free();
}
// bonus point*: is this okay to do after the Free() call?
int test = reply.OtherIrrelevantData;
return result_buffer;
}
While this works correctly, I would like to know if this is the most efficient / most correct way of implementing this function.
Is there some method converting a managed byte array to a managed structure that doesn't involve an intermediate native handle and a copy? For instance, in C++, I would just do this:
NATIVE_METHOD_REPLY* reply = reinterpret_cast< NATIVE_METHOD_REPLY* >( reply.DataPtr );
*For a bonus point, is it okay to use data in the structure after the native handle has been freed?
Thanks,
PaulH
Edit: Updated solution
[DllImport("my_lib.dll", SetLastError = true)]
internal static extern Int32 Foo(IntPtr replyBuffer, Int32 replySize);
public byte[] void Bar()
{
byte[] result_buffer = new byte[256];
int reply_buffer_len = Marshal.SizeOf(typeof(NativeMethods.NATIVE_METHOD_REPLY)) + result_buffer.Length;
IntPtr reply_buffer = Marshal.AllocCoTaskMem(reply_buffer_len);
NativeMethods.NATIVE_METHOD_REPLY reply;
try
{
NativeMethods.Foo(reply_buffer, reply_buffer_len);
reply = (NativeMethods.NATIVE_METHOD_REPLY)Marshal.PtrToStructure(
reply_buffer,
typeof(NativeMethods.NATIVE_METHOD_REPLY));
Marshal.Copy(reply.DataPtr, result_buffer, 0, reply.DataSize);
}
finally
{
Marshal.FreeCoTaskMem(reply_buffer);
}
return result_buffer;
}
The structure has a fixed size. There's no point in passing an array, just pass the structure:
[DllImport("my_lib.dll", SetLastError = true)]
internal static extern Int32 Foo(out NATIVE_METHOD_REPLY replyBuffer, Int32 replySize);
You do have a memory management problem. Who owns the pointer?
Okay, the structure is actually variable sized and the pointer points into the array. You need nother approach. Simply allocate a chunk of unmanaged memory up front instead of letting the P/Invoke marshaller copy the data into a managed array. Which is in fact a hard requirement since the garbage collector can move the array, invalidating the pointer. Call Marshal.CoTaskMemAlloc() to reserve the memory, you'll have to free it later. And change the first argument of the function to IntPtr (not out).
You'll find that marshaling the structure a lot easier too, no need to pin the memory. Don't forget to Marshal.FreeCoTaskMem() when you're done.
In C# under the full framework, you can marshal the array directly. See Default Marshaling for Arrays. I don't know what the limitations are on the Compact Framework.
In C# PInvoke, how do I pass a string buffer so that the C DLL fills it and returns? What will be the PInvoke declaration?
The C function declaration is
int GetData(char* data, int buflength);
In C#, I have declared it as
[DllImport(DllName)]
static extern Int32 GetData([MarshalAs(UnmanagedType.LPStr)]StringBuilder receiveddata, Int32 buflen);
Is it correct? I'm passing the StringBuilder variable like this
int bufferLength = 32;
StringBuilder data = new StringBuilder(bufferLength);
int result = GetData(data, bufferLength);
I would like to know is it correct or not?
Thanks
I believe it's correct.
[DllImport(DllName)]
static extern int GetData(StringBuilder data, int length);
which is called like this:
StringBuilder data = new StringBuilder(32);
GetData(data, data.Capacity);
I once wanted to have more control over the bytes returned by my function and did it like this:
[DllImport(DllName)]
private unsafe static bool GetData(byte* data, int length);
used like this:
byte[] bytes = new byte[length];
fixed(byte* ptr = bytes)
{
bool success = Library.GetData(ptr, length);
if (!success)
Library.GetError();
return Encoding.UTF8.GetString(bytes);
}
I don't think that using MarshalAs attribute necessary here. StringBuilder is a right choice for char* out.
I guess it will be good to add the CharSet property since you are dealing with strings here.
Like this:
[DllImport(DllName, CharSet=CharSet.Auto)]