Here is a C# program that tries Marshal.SizeOf on a few different types:
using System;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Sequential)]
class AClass { }
[StructLayout(LayoutKind.Sequential)]
struct AStruct { }
[StructLayout(LayoutKind.Sequential)]
class B { AClass value; }
[StructLayout(LayoutKind.Sequential)]
class C<T> { T value; }
class Program
{
static void M(object o) { Console.WriteLine(Marshal.SizeOf(o)); }
static void Main()
{
M(new AClass());
M(new AStruct());
M(new B());
M(new C<AStruct>());
M(new C<AClass>());
}
}
The first four calls to M() succeed, but on the last one, SizeOf throws an ArgumentException:
"Type 'C`1[AClass]' cannot be marshaled as an unmanaged structure; no meaningful size or offset can be computed."
Why? Specifically, why does SizeOf choke on C<AClass>, but not on B or on C<AStruct>?
EDIT: Because it was asked about in the comments, here's the "real-world" problem that inspired this mostly-academic question:
I'm calling into a C API that is basically one C function that operates on (pointers to) lots of different types of simple C structures. All contain a common header followed by one field, but the type of that field is different in different structures. A flag in the header indicates the type of the field. (Strange, yes, but that's what I have to work with).
If I could define a single generic type C<T> and a single C# extern declaration M(C<T>), and then call M(C<int>) on one line, and M(C<double>) on another, I'd have a short and sweet interop solution. But given JaredPar's answer, it appears that I have to make a separate C# type for each structure (though inheritance can provide the common header).
Generics as a rule are not supported in any interop scenario. Both P/Invoke and COM Interop will fail if you attempt to marshal a generic type or value. Hence I would expect Marshal.SizeOf to be untested or unsupported for this scenario as it is a marshal-specific function.
It is not known what size the aggregated object T would have (it would be the size of a pointer if T is a referenece type and mostly any value if it is value type).
I think you can solve this problem by setting the MarshalAs attribute on the field 'value' specifying the most matching type (for example, Unmanagedtype.SysInt). Note that it still won't work for so-called non-mappable types (i.e. types for which fields offsets and sizes can't be deduced easily).
But AFAIK, it's not recommended to use generics in interop.
Related
I have a device that transmits binary data. To interpret the data I have defined a struct that matches the data format. The struct has a StuctLayoutAttribute with LayoutKind.Sequential. This works as expected, e.g:
[StructLayout(LayoutKind.Sequential)]
struct DemoPlain
{
public int x;
public int y;
}
Marshal.OffsetOf<DemoPlain>("x"); // yields 0, as expected
Marshal.OffsetOf<DemoPlain>("y"); // yields 4, as expected
Marshal.SizeOf<DemoPlain>(); // yields 8, as expected
Now I wish to treat one struct similar to an other struct, so I experimented with the structure implementing an interface:
interface IDemo
{
int Product();
}
[StructLayout(LayoutKind.Sequential)]
struct DemoWithInterface: IDemo
{
public int x;
public int y;
public int Product() => x * y;
}
Marshal.OffsetOf<DemoWithInterface>("x").Dump(); // yields 0
Marshal.OffsetOf<DemoWithInterface>("y").Dump(); // yields 4
Marshal.SizeOf<DemoWithInterface>().Dump(); // yields 8
To my surprise the offsets and size of DemoWithInterface remain the same as DemoPlain and converting the same binary data from the device to either an an array of DemoPlain or an array of DemoWithInterface both work. How is this possible?
C++ implementations often use a vtable (see Where in memory is vtable stored?) to sore virtual methods. I believe that in C# methods published in an interface, and methods that are declared virtual, are similar to virtual methods in C++ and that it requires something similar to a vtable to find the correct method. Is this correct or does C# do it completely different? If correct, where is the vtable like structure stored? If different, how is C# implemented with respect to interface inheritance and virtual methods?
Basically, "does not apply". Structs in C# - as has been discussed - do not support inheritance, and so no v-table is required.
The field layout is the field layout. It is simply: where the actual fields are. Implementing interfaces doesn't change the fields at all, and doesn't require any change to the layout. So that's why the size and layout isn't impacted.
There are some virtual methods that structs can (and usually should) override - ToString() etc. So you can legitimately ask "so how does that work?" - and the answer is: smoke and mirrors. Also known as constrained call. This defers the question of "virtual call vs static call" to the JIT. The JIT has full knowledge as to whether the method is overridden or not, and can emit appropriate opcodes - either a box and virtual call (a box is an object, so has a v-table), or a direct static call.
It might be tempting to think that the compiler should do this, not the JIT - but often the struct is in an external assembly, and it would be catastrophic if the compiler emitted a static call because it could see the overridden ToString() etc, and then someone updates the library without rebuilding the app, and it gets a version that doesn't override (MissingMethodException) - so constrained call is more reliable. And doing the same thing even for in-assembly types is just simpler and easier to support.
This constrained call also happens for generic (<T>) methods - since the T could be a struct. Recall that the JIT executes per T for value-typed T on a generic method, so it can apply this logic per-type, and bake in the actual known static call locations. And if you're using something like .ToString() and your T is a struct that doesn't override that: it will box and virtual-call instead.
Note that once you assign a struct to an interface variable - for example:
DemoWithInterface foo = default;
IDemo bar = foo;
var i = bar.Product();
you have "boxed" it, and everything is now virtual-call on the box. A box has a full v-table. That is why generic methods with generic type constraints are often preferable:
DemoWithInterface foo = default;
DoSomething(foo);
void DoSomething<T>(T obj) where T : IDemo
{
//...
int i = obj.Product();
//...
}
will use constrained call throughout and will not require a box, despite accessing interface members. The JIT resolves the static call options for the specific T at execution.
Default Marshalling Behavior | Microsoft Docs and especially the section Value Types Used in Platform Invoke provides the answer:
When marshaled to unmanaged code, these formatted types are marshaled as C-style structures.
and
When a formatted type is marshaled as a structure, only the fields within the type are accessible. If the type has methods, properties, or events, they are inaccessible from unmanaged code.
So the C# struct's (virtual) methods are stripped and only a plain C-struct in transmitted. In the OP's case the device sends the bytes which comprise a plain C-struct, Marshal.PtrToStructure<T>(IntPtr) converts the bytes to a C#-struct and, in case of DemoWithInterface, attaches the Product-method and the vtable (or other meta-data) to make the struct implement IDemo.
I have been trying to create a handle to a structure type because I need a pinned pointer to it, but I am getting the error "Object contains non-primitive or non-blittable data"
My structure looks like this:
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.U1)]
public bool Test;
}
Now, when I call,
var mystruct = new MyStruct();
var handle = GCHandle.Alloc(mystruct, GCHandleType.Pinned);
I get the error "Object contains non-primitive or non-blittable data". Now I understand that the bool field is a non-blittable type. But I was under the impression that by adding the MarshalAs attribute, I could tell the marshaller how to convert the type. (I also tried UnmanagedType.Bool)
This structure has to be defined globally, because it is needed throughout my class. The only reason I need the pointer is because I have an unmanaged API that must pass this structure as a pointer. Then I have to get that structure in a callback and read/update members.
So this is the basic scenario.
Structure is created globally in a managed class
Pointer to structure is obtained
Pointer to the structure is passed into the API
The API calls a static method callback where I then need to get my structure and read/update members.
I tried to use Marshal.StructureToPtr but this only creates a copy, so if in my managed class I update the member, when the callback is raised, the updated value is not there.
Does anyone know how I can get a pinned pointer to my structure so I can read/modify the public members and have them available in the callback?
Thanks
You've got more than one problem here. Using a struct is highly inadvisable. It will get boxed before the GCHandle.Alloc() call and that boxed object gets pinned. You cannot see any updates to it through your mystruct variable. Use a class instead.
And avoid bool, it is a non-blittable type due to its highly variable implementation. It is 4 bytes in C, 1 byte in C++, 2 bytes in COM. Just make it a byte instead. You can write a property to get it back to a bool.
So:
[StructLayout(LayoutKind.Sequential)]
public class MyStruct
{
private byte _test;
public bool Test {
get { return _test != 0; }
set { _test = value ? 1 : 0; }
}
}
You're right that you're telling the marshaller how to marshal the type.
But that won't do you any good when you then attempt to bypass the marshaller.
You need to decide whether you want to use the marshaller, or whether you want the unmanaged code to directly write into managed memory.
If you want to use the marshaller:
Generally, a good way to handle this is to use it in both directions. You can use Marshal.StructureToPtr (as you've found), call the external function, and then use Marshal.PtrToStructure to convert it back into your managed representation.
Or you can use methods that are set up in such a way that marshalling happens automatically, without you needing to specify this manually. For example, calling a native method taking a ref MyStruct parameter will allow for that to happen.
If you don't want to use the marshaller:
Don't use any types that require marshalling. As Hans Passant comments, use a different type instead, byte would probably be a good choice.
(I'll refrain from commenting on the advantages and disadvantages of using structs here, except that the points already made about it are well worth reading and understanding.)
Here is a simple C code (VS 2013 C++ project, "compiled as C"):
typedef struct {
bool bool_value;
} BoolContainer;
BoolContainer create_bool_container (bool bool_value)
{
return (BoolContainer) { bool_value };
}
Here is my P/Invoke wrapper
public partial class NativeMethods
{
[DllImport(LibraryName, CallingConvention = CallingConvention.Cdecl)]
public static extern BoolContainer create_bool_container([MarshalAs(UnmanagedType.I1)] bool bool_value);
}
Here are the managed versions of the BoolContainer:
First is throwing MarshalDirectiveException: Method's type signature is not PInvoke compatible.:
public struct BoolContainer // Marshal.SizeOf(typeof(BoolContainer)) = 1
{
[MarshalAs(UnmanagedType.I1)] // same with UnmanagedType.U1
public bool bool_value;
}
Second is working:
public struct BoolContainer // Marshal.SizeOf(typeof(BoolContainer)) = 1
{
public byte bool_value;
}
Test code:
BoolContainer boolContainer = NativeMethods.create_bool_container(true);
Is seems like DllImport ignores MarshalAs for any boolean members in the returned structure. Is there a way to keep bool in the managed declaration?
Structs as return types are pretty difficult in interop, they don't fit a CPU register. This is handled in native code by the caller reserving space on its stack frame for the return value and passing a pointer to the callee. Which then copies the return value through the pointer. In general a pretty troublesome feature, different C compilers have different strategies to pass that pointer. The pinvoke marshaller assumes MSVC behavior.
This is a problem with structs that are not blittable, in other words when its layout is no longer an exact match with the unmanaged struct that the native code needs. That requires an extra step to convert the unmanaged struct to the managed struct, the equivalent of Marshal.PtrToStructure(). You see this for example if you add a string field to your struct. That makes it non-blittable since the C string has to be converted to a System.String, you'll get the exact same exception.
A similar kind of problem in your case, bool is also a very difficult type to interop with. Caused by the C language not having the type and it getting added later with highly incompatible choices. It is 1 byte in C++ and the CLR, 2 bytes in COM, 4 bytes in C and the winapi. The pinvoke marshaller picked the winapi version as the default, making it match BOOL, the most common reason to pinvoke. This forced you to add the [MarshalAs] attribute and that also requires extra work, again to be done by the equivalent of Marshal.PtrToStructure().
So what the pinvoke marshaller is missing is the support for this extra step, custom marshaling of the return value after the call. Not sure if this was a design constraint or a resource constraint. Probably both, ownership of non-blittable struct members is completely unspecified in unmanaged languages and guessing at it, especially as a return value, very rarely turns out well. So they may well have made the call that working on the feature just wasn't worth the low odds of success. This is a guess of course.
Using byte instead is a fine workaround.
In C# we have a datatype object which can hold any type of data. Same thing I want to achieve in VC++. Can anyone kindly let me know VC++ equivalent of "Object of C#".
IN C#, in the calling appl program (say call.cs)
object ob=null;
ob=(object)str;
funct(ref ob);
Here str is empty string.
This thing I want to achieve in VC++. So I need to create VC++ equivalent of object.
I am sure we need to use pointers as ref's equivalent??
There isn't one. C++ doesn't have a unified type hierarchy like .NET languages have.
The closest you can get is a void* (pointer-to-void), which can point to any type of object. You should avoid void*s like the plague, though; once you start using them you lose any and all type safety.
As other commentators have said, C++ does not have a common base-class for every object. Theoretically, you could create your own and derive everything from it:
class Object
{
protected:
Object(){};
virtual ~Object(){};
public:
virtual std::string toString() const {return("");};
}; // eo class Object
This, however, won't help you with integral types such as int, short. You'd have to make your own:
class Int : public Object
{
private:
int m_nVal;
public:
Int(int _val = 0) : m_nVal(_val){};
Int(const Int& _rhs) : m_nVal(_rhs.m_nVal){};
virtual ~Int(){};
// operators
operator int() const {return(m_nVal);}
bool operator == (const Int& _rhs) const {return(m_nVal == _rhs.m_nVal);};
bool operator == (int _val) const {return(m_nVal == _val);};
Int& operator = (const Int& _rhs) {m_nVal = _rhs.m_nVal; return(*this);}:
Int& operator = (int _val) {m_nVal = _val; return(*this);};
// .... and all the other operators
// overrides
virtual std::string toString() const
{
std::ostringstream oss;
oss << m_nVal;
return(oss.str());
};
}; // eo class Int
You'd then have to do this for all the other types you want to use. Once done you can pass them around as if they were ints, bools, longs etc (thanks to operator overloading). A better method would be to use a template class for the integral types:
template<class T> class IntegralType : public Object
{
private:
T m_Val;
public:
// rest of class looks the same
}; // eo class IntegralType<>
Then typedef them away:
typedef IntegralType<int> Int;
typedef IntegralType<short> Short;
typedef IntegralType<long> Long;
Even using a template-class like this to take the leg-work out of it, you'd still need a specialisation for strings/bools. implementing operator ++ on IntegralType<> will work fine for numbers, but is going to throw up on std::string.
If you went the template route, you've now got "Object", integral types and some specialisations for strings, bools. But to mimick .NET even more, you probably want to introduce interfaces for comparisons:
template<class T> class IEquitable
{
public:
virtual Equals(T _other) = 0;
}; // eo class IEquitable<>
That can easily be plumbed in to your IntegralType<> classes and the specialisations.
But as another commentator pointed out, why would you? boost::any is useful if you're trying to do something like a Tuple which has a name and a value of an arbitrary type. If you need to build a collection of these then there is something fundamentally wrong with your design. For example, in all my coding in C# I have never had to write:
List<Object> list = new List<Object>();
There may have been:
List<Vehicle> list;
List<Employee> List;
Dictionary<string, Alien> aliens;
But never anything at the Object level. Why? Well apart from calling ToString() on it, or perhaps doing some risky casting, why would you want to? Generics exist in programming so that we do not have to have lists of objects (or in the case of C++, void*).
So there you have it. The above shows how you might have objects and integral types working kind of like C#, and I've missed a chunk of stuff out. Now it's time to look at your design and decide if that's what you really need to do.
There's nothing built into the language. Usually, wanting it at all indicates that your design isn't very well thought out, but if you can't figure out any alternative, you might consider (for one example) Boost any.
The <comutil.h> header contains a handy wrapper for VARIANT. Takes care of proper initialization and cleanup.
#include <comutil.h>
#ifdef _DEBUG
# pragma comment(lib, "comsuppwd.lib")
#else
# pragma comment(lib, "comsuppw.lib")
#endif
...
_variant_t arg = L"some string";
someComPtr->func(&arg);
There isn't anything in your code snippet that would help me help you figuring out how to obtain the COM interface pointer. Start a new question about that if you have trouble.
The alternative for you is to look at System.Runtime.InteropServices.GCHandle, which allows you to find the managed object from unmanaged code, but in any way you will end up with nasty and risky type casts and you need to be really careful to keep somewhere a reference to the managed object as it might get garbage-collected if there is only a reference in unmanaged code.
I am writing a C wrapper an would like to use a typedef aquivalent to define some types which should be valid in quite a lot of source files. Those "types" are just different aliases to [u]int16/32/64_t, but are useful to distinguish function parameters.
One could use using MyId=System.Int32;, but this needs to be redeclared in every file as far as I see... Is there a better way?
One alternate approach is to use a struct with implicit conversion to the underlying type.
public struct MyHandle
{
private int _handle;
internal MyHandle(int handle)
{
_handle = handle;
}
public static implicit operator int(MyHandle handle)
{
return handle._handle;
}
}
Your internal code can still use it as the underlying type (int in this case) via the implicit conversion, but you expose it as a strong type to the user. Your user can also see the int value, though it's effectively meaningless to them. They can't directly cast an int to your Handle type, as the constructor is internal to your assembly and you don't provide a conversion operator for the other direction.
Since I'm assuming that you want to distinguish between a legitimate use of say UInt32 and your custom type, yes you would need to manually reference your alias everywhere you want it to be used.