I have the following marshalling code in my project. I have few questions on this.
[DllImport=(Core.dll, SetLastError=true, EntryPoint="CoreCreate", CharSet="CharSet.Ansi", CallingConvention="CallingConvention.Cdecl")]
internal static extern uint CoreCreate(ref IntPtr core);
Why 'internal static extern' is required? Is this compulsory? Why this is used?
What is SetLastError?
[StructLayout(LayoutKind.Sequential,CharSet=CharSet.Ansi)]
internal struct Channel
{
internal byte LogicalChannel;
}
Why LayoutKind.Sequential?
Why 'internal static extern' is required?
The internal modifier just set visibility of your method. It's not required to be internal so you can declare the method private or public as you need and as you would do with any other standard method.
The static modifier is required because it's not an instance method and that method doesn't know any class (it hasn't a this pointer).
Finally extern is required to inform the compiler that the method isn't implemented here but in another place (and you'll specify where using attributes). Evey extern method must be declared static too (because it's a simple function call without any knowledge about objects).
What is SetLastError?
It indicates that method may change the thread's last-error code value. See the GetLastError() function for details about this. If the called function will change this value then it's a good thing to set SetLastError to true, from MSDN:
The runtime marshaler calls GetLastError and caches the value returned to prevent it from being overwritten by other API calls. You can retrieve the error code by calling GetLastWin32Error.
In short it saves the value returned by GetLastError() to an internal cache so any other call to system API (even internal to others framework functions) won't overwrite that value.
Why LayoutKind.Sequential?
Class layout in .NET isn't required to be sequential in memory (sequential = if A is declared before B then memory layout has A before B). This is not true in C where the declaration order matters (declaration is used by the compiler to understand the layout, in memory, of raw data). If you have to interop with C functions then you have to be sure about the layout of the data you pass them. This is how LayoutKind.Sequential works: it instructs the compiler to respect the declaration sequential order for data in the struct. This is not the only option to interop with unmanaged world, you can even explicitly set the offset (from structure beginning) of each field (see LayoutKind.Explicit).
This is not an answer, just a few comments:
"internal static" is one thing and "extern" is another thing needed when calling external dll's.
SetLastError or GetLastError is methods we used a lot in the "old" days to get error messages from windows about the latest handling.
LayoutKind.Sequential is a way to inform the compiler to layout the struct in a specified way - you may need to do this if marchalling to other systems.
Related
If I have a native code that expects a structure with two fields:
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern int my_method(ref MyStruct myStruct);
// this is what native code expects
public struct MyStruct {
IntPtr First;
IntPtr Second;
}
but instead I pass another struct to it, will it work or not - by design or by accident?
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern int my_method(ref MyLongerStruct myLongerStruct);
// this is what I want to pass to it
public struct MyLongerStruct {
IntPtr First;
IntPtr Second;
object ObjectPointer;
}
Will the object reference added to the end of the struct at C# side somehow affect P/Invoke call?
I shouldn't work. And even more, you need to add and properly set StructLayoutAttribute to the structure, as it explained here
I think, the result should be like this:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct {
IntPtr First;
IntPtr Second;
}
If the total difference in structure is fields added to the end, and you use StructLayout to prevent the compiler from optimizing the memory layout of your struct (as Alex Butenko suggests), then it's unlikely that there will be any negative side effects apart from a slight speed hit.
When you pass a managed struct to an external function via P/Invoke (using the DllImport attribute) there is a marshaling phase that converts your struct to a compatible format for the target. For ref and out parameters the temporary is converted back when the invoked function returns, copying the values back to your struct instance. All of this is abstracted away, although exactly how the marshaling is performed for each member can be tweaked with the right attributes.
This is how the .NET framework handles strings in P/Invoke. Since it can't just send a string instance pointer to an API function that is expecting a char * (the two are nothing alike) there has to be some translation.
The fun part is that the marshaling code doesn't know anything about what the target is expecting other than what you tell it at the C# end, so if you are sending an extended version of the structure it will do the whole thing. At the other end the native code will get a pointer to a memory block containing the information it's expecting, and it won't have any way to tell that there is more after the end of the structure.
Apart from that, no problem... as long as you're passing by reference and not by value. Passing structs by value is something that should raise big red stop signs all over your brain. Don't do it, it's evil.
I have been trying to create a handle to a structure type because I need a pinned pointer to it, but I am getting the error "Object contains non-primitive or non-blittable data"
My structure looks like this:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.U1)]
public bool Test;
}
Now, when I call,
var mystruct = new MyStruct();
var handle = GCHandle.Alloc(mystruct, GCHandleType.Pinned);
I get the error "Object contains non-primitive or non-blittable data". Now I understand that the bool field is a non-blittable type. But I was under the impression that by adding the MarshalAs attribute, I could tell the marshaller how to convert the type. (I also tried UnmanagedType.Bool)
This structure has to be defined globally, because it is needed throughout my class. The only reason I need the pointer is because I have an unmanaged API that must pass this structure as a pointer. Then I have to get that structure in a callback and read/update members.
So this is the basic scenario.
Structure is created globally in a managed class
Pointer to structure is obtained
Pointer to the structure is passed into the API
The API calls a static method callback where I then need to get my structure and read/update members.
I tried to use Marshal.StructureToPtr but this only creates a copy, so if in my managed class I update the member, when the callback is raised, the updated value is not there.
Does anyone know how I can get a pinned pointer to my structure so I can read/modify the public members and have them available in the callback?
Thanks
You've got more than one problem here. Using a struct is highly inadvisable. It will get boxed before the GCHandle.Alloc() call and that boxed object gets pinned. You cannot see any updates to it through your mystruct variable. Use a class instead.
And avoid bool, it is a non-blittable type due to its highly variable implementation. It is 4 bytes in C, 1 byte in C++, 2 bytes in COM. Just make it a byte instead. You can write a property to get it back to a bool.
So:
[StructLayout(LayoutKind.Sequential)]
public class MyStruct
{
private byte _test;
public bool Test {
get { return _test != 0; }
set { _test = value ? 1 : 0; }
}
}
You're right that you're telling the marshaller how to marshal the type.
But that won't do you any good when you then attempt to bypass the marshaller.
You need to decide whether you want to use the marshaller, or whether you want the unmanaged code to directly write into managed memory.
If you want to use the marshaller:
Generally, a good way to handle this is to use it in both directions. You can use Marshal.StructureToPtr (as you've found), call the external function, and then use Marshal.PtrToStructure to convert it back into your managed representation.
Or you can use methods that are set up in such a way that marshalling happens automatically, without you needing to specify this manually. For example, calling a native method taking a ref MyStruct parameter will allow for that to happen.
If you don't want to use the marshaller:
Don't use any types that require marshalling. As Hans Passant comments, use a different type instead, byte would probably be a good choice.
(I'll refrain from commenting on the advantages and disadvantages of using structs here, except that the points already made about it are well worth reading and understanding.)
Given the following c++ class in foo.dll
class a{
private:
int _answer;
public:
a(int answer) { _answer = answer; }
__declspec(dllexport) int GetAnswer() { return _answer; }
}
I would like the pInvoke GetAnswer from C#. To do that, I use the following method:
[DllImport("foo.dll", CallingConvention = CallingConvention.ThisCall, EntryPoint= "something")]
public static extern int GetAnswer(IntPtr thisA);
And I pass in an IntPtr that points to an a (that I got from somewhere else, it's not important). CallingConvention = CallingConvention.ThisCall makes sure it's handled correctly
What's cool about this question is that I know I'm right so far because it's already working great! Using Depends.exe, I can see that "GetAnswer" is exported as ?GetAnswer#a##UAEHXZ (Or something close - the point being that it's been name mangled). When I plug the mangled name into the "something" for the EntryPoint everything works great! It took me about a day before it dawned on me to use Depends.exe, so I'm going to leave this here as a help to anybody who has a similar issue.
My REAL Question is: Is there any way to disable C++ name mangling on GetAnswer so that I don't need to put the mangled name in as my entry point. Having the mangled name in there seems like it could break, because my understanding of name mangling is that it can change if the compiler changes. Also it's a pain in the butt to use Depends.exe for every instance method that I want to pInvoke.
Edit: Forgot to add what I've tried:
I don't seem to be able to put extern "C" on the function declaration, although I can stick it on the definition. This doesn't seem to help though (which is obvious when you think about it)
The only other solution I can think of is a c-style function that wraps the instance method and takes an instance of an a as a parameter. Then, disable name mangling on that wrapper and pInvoke that. I'd rather stick with the solution that I already have, though. I already told my co-workers that pInvoke is great. I'm going to look like an idiot if I have to put special functions in our c++ library just to make pInvoke work.
You cannot disable mangling for a C++ class method, but you may well be able to export the function under a name of your choice using /EXPORT or a .def file.
However, your entire approach is brittle because you rely on an implementation detail, namely that this is passed as an implicit parameter. And what's more, exporting individual methods of a class is a recipe for pain.
The most sensible strategies for exposing a C++ class to .net languages are:
Create flat C wrapper functions and p/invoke those.
Create a C++/CLI mixed mode layer that publishes a managed class that wraps the native class.
Option 2 is preferable in my opinion.
You may be able to use the comment/linker #pragma to pass the /EXPORT switch to the linker which should allow you to rename the exported symbol:
#pragma comment(linker, "/EXPORT:GetAnswer=?GetAnswer#a##UAEHXZ")
Unfortunately, this does not resolve your need to look up the mangled name using depends or some other tool.
You do not have to disable the mangled name which actually contains lots of information of how the function itself is declared, it basically represents the whole signature of the function after the function name gets de-mangled. I understand you already found a word-around and the other answer has been marked as a correct answer. What I am writing below is how we can make it work as you desired.
[DllImport("foo.dll", CallingConvention = CallingConvention.ThisCall, EntryPoint = "#OrdinalNumber")]
public static extern int GetAnswer(IntPtr thisA);
If you replace "#OrdinalNumber" with the real ordinal number of GetAnsweer, such as "#1", it will work as you desired.
You may just consider the EntryPoint property is the same as the function name we pass to GetProcAddress where you can either pass the function name or the ordinal number of the function.
Your approach to calling non-static function members of a C++ class is indeed correct and thiscall is used correctly and that is exactly thiscall calling convention comes in play in C# P/Invoke. The issue with this approach is that you will have to look into the DLL's PE information, Export Function Information and find out the ordinal number for each function you would like to call, if you have a big number of C++ functions to call, you may want to automate such a process.
From the Question Author: The solution I actually went with
I ended up going with a c-style function that wraps the instance method and takes an instance of an a as a parameter. That way, if the class ever does get inherited from, the right virtual method will get called.
I deliberately chose not to go with C++/CLI because it's just one more project to manage. If I needed to use all of the methods on a class, I would consider it, but I really only need this one method that serializes the class data.
In another question I asked, a comment arose indicating that the .NET framework's Array.Copy method uses unmanaged code. I went digging with Reflector and found the signature one of the Array.Copy method overloads is defined as so:
[MethodImpl(MethodImplOptions.InternalCall), ReliabilityContract(Consistency.MayCorruptInstance, Cer.MayFail)]
internal static extern void Copy(Array sourceArray, int sourceIndex, Array destinationArray, int destinationIndex, int length, bool reliable);
After looking at this, I'm slightly confused. The source of my confusion is the extern modifier which means (MSDN link):
The extern modifier is used to declare
a method that is implemented
externally.
However, the method declaration is also decorated with a MethodImplOptions.InternalCall attribute, which indicates (MSDN link):
Specifies an internal call. An
internal call is a call to a method
that is implemented within the common
language runtime itself.
Can anyone explain this seemingly apparent contradiction?
I would have just commented on leppie's post, but it was getting a bit long.
I'm currently working on an experimental CLI implementation. There are many cases where a publicly exposed method (or property) can't be implemented without knowledge of how the virtual machine is implemented internally. One example is OffsetToStringData, which requires knowledge of how the memory manager allocates strings.
For cases like this, where there is no C# code to express the method, you can treat each call to the method in a special way internal to the JIT process. As an example here, replacing the call byte code with a ldc.i4 (load constant integer) before passing it to the native code generator. The InternalCall flag means "The body of this method is treated in a special way by the runtime itself." There may or may not be an actual implementation - in several cases in my code the call is treated as an intrinsic by the JIT.
There are other cases where the JIT may have special information available that allows heavy optimization of a method. One example is the Math methods, where even though these can be implemented in C#, specifying InternalCall to make them effectively intrinsics has significant performance benefits.
In C#, a method has to have a body unless it is abstract or extern. The extern means a general "You can call this method from C# code, but the body of it is actually defined elsewhere.". When the JIT reaches a call to an extern method, it looks up where to find the body and behaves in different ways per the result.
The DllImport attribute instructs the JIT to make a P/Invoke stub to call a native code implementation.
The InternalCall flag instructs the JIT to treat the call in a self-defined way.
(There are some others, but I don't have examples off the top of my head for their use.)
InternalCall means provided by the framework.
extern says you are not providing code.
extern can be used in 2 general situations, like above, or with p/invoke.
With p/invoke, you simply tell the method where to get the implementation.
I want to ask why all extern method calls are static?
How the CLR handles these calls?
Extern method calls are to unmanaged code. As such, it doesn't make sense to be called on a (managed) object instance - the first (hidden) argument in an instance method is the instance reference, aka this. Typically, extern methods just involve simple types (primitives, string, etc) - not objects (except perhaps arrays - and even they are often resolved to IntPtr first).
extern calls also must generally conform to a "C-style" API, and C doesn't know anything about objects, thus the calls are static.
My statement isn't 100% true as there is a ThisCall calling convention which can be used with [DllImport] as an aid in calling C++ methods.