I am using PInvoke to call a C++ function from my C# program. The code looks like this:
IntPtr data = Poll(this.vhtHand);
double[] arr = new double[NR_FINGERS /* = 5 */ * NR_JOINTS /* = 3*/];
Marshal.Copy(data, arr, 0, arr.Length);
With Poll()'s signature looking like this:
[DllImport("VirtualHandBridge.dll")]
static public extern IntPtr Poll(IntPtr hand);
The C-function Poll's signature:
extern "C" __declspec(dllexport) double* Poll(CyberHand::Hand* hand)
Unless I'm having a huge brain failure (admittedly, fairly common for me), this looks to me like it should be working.
However, the double values I am getting are completely incorrect, and I think this is because of incorrect memory usage. I have looked it up, and I think doubles in C# and C++ are identical in size, but maybe there is some other issue playing here. One thing that rubs me the wrong way is that Marshal.Copy is never told what type of data it should expect, but I read that it is supposed to be used this way.
Any clues, anyone? If needed, I can post the correct results and the returned results.
You are missing the CallingConvention property, it is Cdecl.
You really want to favor a better function signature, the one you have is extremely brittle due to the memory management problem, the required manual marshaling, the uncertainty of getting the right size array and the requirement to copy the data. Always favor the caller passing a buffer that your native code fills in:
extern "C" __declspec(dllexport)
int __stdcall Poll(CyberHand::Hand* hand, double* buffer, size_t bufferSize)
[DllImport("foo.dll")]
private static extern int Poll(IntPtr hand, double[] buffer, int bufferSize)
Use the int return value to report a status code. Like a negative value to report an error code, a positive value to return the number of elements actually copied into the buffer.
You shouldn't even need to marshal the data like that, as long as you declare the P/Invoke correctly.
If your CyberHand::Hand* is in reality a pointer to a double, then you should declare your P/Invoke as
[DllImport("VirtualHandBridge.dll")]
static public extern IntPtr Poll(double[] data);
And then just call it with your array of doubles.
If it isn't really an array of doubles, then you certainly can't do what you're doing.
Also, how does your 'C' function know how big the array will be? Is it a fixed size?
The IntPtr return value will be a problem. What is the double* pointing to? An array or a single item?
You could find that it's easier (if you can) to write a simpler more friendly 'C' wrapper for the function you're calling, and call the wrapper function itself. You can of course only do that if you can change the source code of the 'C' DLL. But without knowing exactly what your function does, I can't give you specific advice.
[EDIT]
Ok, your code should theoretically work if the memory being passed back isn't being messed around with (e.g. freed up). If it's not working, then I suspect something like that is happening. You'd definitely be better writing a wrapper 'C' function that fills in an array allocated by the C# and passed to the function, rather than passing back a pointer to some internal memory.
BTW: I don't like code which passes around pointers to blocks of memory without also passing the size of that block. Seems a bit prone to nasty things.
Related
I have an array of arrays of this struct (shown here in C#, but existing in C++ as well):
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
IntPtr name; //pointer to string, char* on C++ side
long pValues;
long jValues;
long eValues;
long kValues;
int cost;
};
and an algorithm in a C++ DLL that does work on it, being called from managed C# code. It's CPU-heavy, which is what necessitates this as it runs much faster in C++ than C#. The managed (C#) side never has to know the contents of the struct data, as the algorithm only returns a single array of ints.
So, how would I go about storing this data in the most efficient way (ie with the least overhead), for the lifetime of the application? I think I have it narrowed down to two options:
Initialize structs and set values in C#, pin memory with GCHandle and pass reference to C++ whenever I want to do work (see this post on Unity forums)
Initialize structs and set values in C++, have structs persist in memory on unmanaged side
So my questions are very specific:
With 1, I'm confused as to how marshalling works. It looks like in MSDN: Copying and Pinning that you are able to pass arrays of structures by pinning and passing a reference to the pinned data, without having to copy or convert any of it (and as long as the struct looks the same on both sides). Am I reading that correctly, is that how it actually works? Referring to the Unity3d forum post, I see Marshal.PtrToStructure being called; I thought that performs copying operations? As the data would be stored on the managed side in this instance, having to copy and/or convert the data every time the C++ function is called would cause a lot of overhead, unless I'm thinking that those type of operations are a lot more expensive than they actually are.
With 2, I'm wondering if it's possible to have persistence between C++ calls. To the best of my knowledge, if you're P/Invoking from a DLL, you can't have persistent data on the unmanaged side, so I can't just define and store my struct arrays there, making the only data transferred between managed and unmanaged the int array resulting from the unmanaged algorithm. Is this correct?
Thank you very much for taking the time to read and help!
If the C# code does not need to know the internals of the array and the structure, don't expose it to the C# code. Do all the work on this type in the unmanaged code and avoid marshalling overhead.
Essentially, you want to follow this basic pattern. I'm sure the details will differ, but this should give you the basic concept.
C++
MyStruct* newArray(const int len)
{
return new MyStruct[len];
}
void workOnArray(MyStruct* array, const int len)
{
// do stuff with the array
}
void deleteArray(const MyStruct* array)
{
delete[] array;
}
C#
[DllImport(dllname)]
static extern IntPtr newArray(int len);
[DllImport(dllname)]
static extern void workOnArray(IntPtr array int len);
[DllImport(dllname)]
static extern void deleteArray(IntPtr array);
I've read the various MSDN pages on C++ Interop with P/Invoke here and here but I am still confused.
I have some large arrays of doubles that I need to get into native code, and some resulting arrays that need to get back. I do not know the sizes of the output arrays in advance. For simplicity, I will use only a single array in the example. The platform is x64; I read that marshalling internals are quite different between 32- and 64-bit environments so this might be important.
C#
[DllImport("NativeLib.dll")]
public static extern void ComputeSomething(double[] inputs, int inlen,
[Out] out IntPtr outputs, [Out] out int outlen);
[DllImport("NativeLib.dll")]
public static extern void FreeArray(IntPtr outputs);
public void Compute(double[] inputs, out double[] outputs)
{
IntPtr output_ptr;
int outlen;
ComputeSomething(inputs, inputs.Length, out output_ptr, out outlen);
outputs = new double[outlen];
Marshal.Copy(output_ptr, outputs, 0, outlen);
FreeArray(output_ptr);
}
C++
extern "C"
{
void ComputeSomething(double* inputs, int input_length,
double** outputs, int* output_length)
{
//...
*output_length = ...;
*outputs = new double[output_length];
//...
}
void FreeArray(double* outputs)
{
delete[] outputs;
}
}
It works, that is, I can read out the doubles I wrote into the array on the C++ side. However, I wonder:
Is this really the right way to use P/Invoke?
Aren't my signatures needlessly complicated?
Can P/Invoke be used more efficiently to solve this problem?
I believe I read that marshalling for single dimensional arrays of built-in types can be avoided. Is there a way around Marshal.Copy?
Note that we have a working C++/Cli version, but there are some problems related to local statics in third-party library code that lead to crashes. Microsoft marked this issue as WONTFIX, which is why I am looking for alternatives.
It is okayish. The complete lack of a way to return an error code is pretty bad, that's going to hurt when the arrays are large and the program runs out of memory. The hard crash you get is pretty undiagnosable.
The need to copy the arrays and to explicitly release them doesn't win any prizes of course. You solve that by letting the caller pass a pointer to its own array and you just write the elements. You however need a protocol to let the caller figure out how large the array needs to be, that is going to require calling the method twice. The first call returns the required size, the second call gets the job done.
A boilerplate example would be:
[DllImport("foo.dll")]
private static int ReturnData(double[] data, ref int dataLength);
And a sample usage:
int len = 0;
double[] data = null;
int err = ReturnData(data, ref len);
if (err == ERROR_MORE_DATA) { // NOTE: expected
data = new double[len];
err = ReturnData(data, len);
}
No need to copy, no need to release memory, good thing. The native code can corrupt the GC heap if it doesn't pay attention to the passed len, not such a good thing. But of course easy to avoid.
If it were practical to separate the code that determines the output length from the code that populates the output then you could:
Export a function that returned the output length.
Call that from the C# code and then allocate the output buffer.
Call the unmanaged code again, this time asking it to populate the output buffer.
But I'm assuming that you have rejected this option because it is impractical. In which case your code is a perfectly reasonable way to solve your problem. In fact I would say that you've done a very good job.
The code will work just the same in x86 once you fix the calling convention mismatch. On the C++ side the calling convention is cdecl, but on the C# side it is stdcall. That doesn't matter on x64 since there is only one calling convention. But it would be a problem under x86.
Some comments:
You don't need to use [Out] as well as out. The latter implies the former.
You can avoid exporting the deallocator by allocating off a shared heap. For instance CoTaskMemAlloc on the C++ side, and then deallocate with Mashal.FreeCoTaskMem on the C# side.
If you knew the array size beforehand, you could write a C++/CLI DLL that takes the managed array as parameter, pins it, and calls the native C++ DLL on the pinned pointer it obtains.
But if it's output-only, I don't see any version without a copy. You can use a SAFEARRAY so P/Invoke does the copying instead of you, but that's all.
I'm writing a C# library where the calling app will pass in a large amount of contiguous, unmanaged memory. This calling app can be either from .Net or Visual C++ (it will go through an intermediate C++/CLI library before calling my library if from C++). It would be useful to validate that there is sufficient memory, so I decided to call the _msize() function. Unfortunately, _msize always seems to give me the wrong size back.
I went back and modified my allocation routine in my sample app and then immediately call _msize. Here is my code snipet:
public unsafe class MyMemory
{
/// <returns></returns>
[DllImport("msvcrt.dll", SetLastError = true)]
public static extern int _msize(IntPtr handle);
public static IntPtr MyAlloc(int size)
{
IntPtr retVal = Marshal.AllocHGlobal(size);
...
int memSize = MyMemory._msize(retVal);
if (memSize < size)
{
...
}
return retVal;
}
When I pass in the size 199229440, I get back memSize of 199178885. I've seen similar results for different numbers. It is less than 0.01% off, which I would totally understand if it was over, but the fact is it is under, meaning _msize thinks the allocated memory is less than what was asked for. Anyone have any clue why this is? And any recommendations on what I should do instead would be appreciated as well.
P/Invoke the LocalSize function instead.
_msize is for determining the size of a block allocated with malloc (and its friends). AllocHGlobal is a wrapper around GlobalAlloc or LocalAlloc (depending on what reference you believe; but I think the two are equivalent), and you want the LocalSize function to determine the size of the block that actually returned. So far as I can tell, Marshal doesn't contain a wrapper for LocalSize, but you can call it using P/Invoke.
So it seems like it's only by sheer good luck that _msize is returning anything useful for you at all. Perhaps malloc uses GlobalAlloc (or LocalAlloc), either always or just when asked for large blocks, and requests a bit of extra space for bookkeeping; in which case _msize would be trying to compensate for that.
I have a C DLL which does some processing on an image and returns the result. So I am trying to pass an image over from the .NET side, but right now I am stuck, and I'm not sure whether that's on the types, the intricacies of marshaling, or syntax. Since I am a noob it could be all three.
I can call other functions in the DLL in question, so at least some of the foundations are in place. For example, when I call:
IntPtr versionIntPtr = GetDLLVersionNumber();
string version = Marshal.PtrToStringAnsi(versionIntPtr);
Console.WriteLine("DLL version number reported as: " + version);
and this works absolutely fine, printing the current version of the DLL to the console.
When I look at C code sample which uses the same function, it appears to use the function as follows:
unsigned char* ExtractImageInfo(const char* pixels, size_t width, size_t height)
(I also have a header file where the same function is mentioned as follows:
... ExtractImageInformation(struct ImageData image, void* imageInformation);
but I don't really know what this means.)
So my attempt to use this function goes as follows:
First I specify the interface to the DLL:
[DllImport("C:\\MyDLL", EntryPoint = "ExtractImageInfo", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr ExtractImageInfo(IntPtr image, UInt32 imageWidth, UInt32 imageHeight);
Then, I get an image of the right type and get its dimensions:
Bitmap bitmap1 = (Bitmap)Image.FromFile("C:\\Images\\myImage.bmp");
UInt32 _imageWidth = Convert.ToUInt32(bitmap1.Width);
UInt32 _imageHeight = Convert.ToUInt32(bitmap1.Height);
Then I get a pointer to the image (since I think that's what I need to pass):
IntPtr bitmap1Ptr = bitmap1.GetHbitmap();
And then I call the function...
IntPtr myProcessedImage;
myProcessedImage = ExtractImageInfo(bitmap1Ptr, _imageWidth, _imageHeight);
But it doesn't work - I don't get back a pointer to my processed data. A big part of what I'm wondering here is whether I am passing the image in the right way, and whether the syntax for my interface to the native function is right. I think probably the answer to both could be NO!
I've only had a few days of reading and experimenting with PInvoke, so if anyone can point out to me the error of my ways I will be eternally grateful :-)
unsigned char* ExtractImageInfo(const char* pixels, size_t width, size_t height)
The name of the argument strongly suggests it wants a pointer to the raw pixel data. You can get one from Bitmap.LockBits(), BitmapData.Scan0 property. Don't call UnlockBits() until the function returns.
The return type of the function is a significant memory management problem. Whatever buffer pointer is returned is almost certainly going to have to be released. You cannot do so from managed code, you don't have access to the allocator used by this C code to call the proper version of free(). Call this function a million times and double-check that you don't have an out-of-control memory leak. If you do then you can't pinvoke it, a C++/CLI wrapper is required although the odds that it works correctly are fairly slim as well. This is a poorly designed function that's hard to use from any code, including C.
Good afternoon all,
I've been working with accessing some external DLLs via the InteropServices.DllImport. I originally settled upon some unsafe code as follows:
internal extern static unsafe void CreateArray(Int32 size, [OutAttribute] UInt32* array);
However, I thought that it might be possible to replace this unsafe code with purely safe code by passing a UInt32 array instead of a pointer. The code changes to
internal extern static void CreateArray(Int32 size, [OutAttribute] UInt32[] array);
which seems to work without any problem. However, I err on the side of caution. Is it possible that the GC may now come along and cause problems? Is there a big difference between passing an array and passing a UInt32 pointer? Are there corner cases I'm missing?
Thanks for your insight,
Giawa
The latter is fine. The P/Invoke layer will pin the managed array in memory while CreateArray is executing, and this approach will require no marshaling, since UInt32[] is a blittable type. This will therefore be just as fast as using a pointer.
With the former declaration you would either have to copy the memory out into a proper managed array, unless you were going to operate on the output entirely through the pointer. And this copying would be more expensive. So, in other words, if you are trying to get a proper managed array out of the call, using the latter syntax will perform better and will require no manual data extraction.