I'm doing interop with some native library, which has some non-natural alignment feature which I want to simulate in .NET struct for the layout. Check these two structs:
public struct Int3
{
public int X;
public int Y;
public int Z;
}
public struct MyStruct
{
public short A;
public Int3 Xyz;
public short B;
}
So, within .NET, it uses its own layout rule to create the layout, which is, alignment would be min(sizeof(primitiveSize), StructLayout.Pack). So the layout of MyStruct would be:
[oo--] MyStruct.A (2 bytes data and 2 bytes padding)
[oooo oooo oooo] MyStruct.Xyz (3 int, no padding)
[oo--] MyStruct.B (2 bytes data and 2 bytes padding)
What I want to do is, I want to change the alignment of Int3 to 8 bytes, like something:
[StructLayout(Alignment = 8)]
public struct Int3 { .... }
Then the layout of MyStruct would became:
[oo-- ----] MyStruct.A (2 bytes for data, and 6 bytes padding, to align next Xyz to 8
[oooo oooo oooo ----] MyStruct.Xyz (4 bytes padding for alignment of 8)
[oo-- ----] (6 bytes padding, because the largest alignment in this struct is 8)
So, my question is:
1) Is there such an attribute in .NET to control the non-natural alignment like this?
2) If there is no such built-in attribute, I know there are other attributes such as StructLayout.Explict, OffsetAttribute, StructLayout.Size, StructLayout.Pack. With these attributes, I can simulate this layout manually, but it is not easy to use. So My second question would be, is there a way to hook into .NET struct layout creation which I can interfere the struct layout? What I mean is, I can create a custom attribute to specify the alignment, and then I calculate the layout, but I don't know how to interfere the .NET to use that layout.
Regards, Xiang.
There is no other way to 'hook into .NET' like you want that I am aware of than StructLayout.Explicit (which is just such a mechanism). Interop is quite a specialized need and, beyond the standard WinAPI cases, you should not expect it to be easy. In your case, unless you are dealing with truly large numbers of different structs with this unusual alignment, it's better to spell it out longhand with StructLayout.Explicit on MyStruct.
Almost any structure will be stored as part of a heap object (either as a class field, or as a field of a struct that is stored as a class field, etc.) The .net 32 platform aligns objects on the Large Object Heap to 16-byte boundaries, but other objects to 4-byte boundaries. Unless an object is manually allocated on the LOH, or is an array of more than 999 doubles [due to a truly atrocious hack, IMHO], there is no meaningful way to assure anything more specific than 4-byte alignment. Even if at some moment in time an unpinned struct is 16-byte aligned, any arbitrary GC cycle might relocate it and change that alignment.
Related
I'm trying to have my C# struct match up some complex padding and packing rules.
Fields should be aligned on 4 byte boundaries.
The entire struct should be a multiple of 16 bytes
Using the StructLayout attribute I can make sure that fields are aligned on 4 byte boundaries.
[StructLayout(LayoutKind.Sequential, Pack=4)]
struct Foo
{
float a;
float b;
}
But looking at the other options for the StructLayout attribute I see no options for padding the struct to multiples of 16 bytes. Is there no option for that in C#?
The only option I see is manually set the right size, using the Size property of the StructLayout attribute. But that seems brittle to me. As every time somebody adds a field to this struct they should take care not to forget to update the size.
After more searching, it indeed does look like I have to manually set the Size and FieldOffset to get to the right packing rules for Constant Buffers in DirectX/HLSL/C# interop code.
In my own project this is even a bit more complex. Since I use source generators to create these structs. But in the end I was able to figure it out. For those interested the source code can be found here.
I came across the following, an excerpt from the book
C# 6.0 in a Nutshell by Joseph Albahari and Ben Albahari (O’Reilly).
Copyright 2016 Joseph Albahari and Ben Albahari, 978-1-491-92706-9.
I do not see it mentioned there, but would it not follow, then, as a rule of thumb - that you should always declare the variables in your type in descending order of size, in order to avoid the memory waste being pointed out ?
I mean, if I got it right, we should all be normally taking that into consideration when designing our types.
Update:
I came up with the snippet below to prove whether the above is right or wrong. I was expecting S2's size to be different, however (to support what the book says).
public struct S1
{
public byte b;
public long l;
}
public struct S2
{
public long l;
public byte b;
}
[StructLayout(LayoutKind.Explicit, Size = 9)]
public struct S3
{
[FieldOffset(0)]
public byte b;
[FieldOffset(1)]
public long l;
}
S1 s1 = new S1();
S2 s2 = new S2();
S3 s3 = new S3();
Console.WriteLine(System.Runtime.InteropServices.Marshal.SizeOf(s1)); //prints 16
Console.WriteLine(System.Runtime.InteropServices.Marshal.SizeOf(s2)); //prints 16
Console.WriteLine(System.Runtime.InteropServices.Marshal.SizeOf(s3)); //prints 9
The book's excerpt is quite misleading. It implies that 7 bytes are 'wasted' but that doesn't have anything to do with the order of the fields. You'd consider swapping the order, putting the long member first. But that doesn't accomplish anything, those 7 bytes are still unused.
Layout and size for a struct is calculated so that the long is still aligned to 8 in an array. In other words, the elements of an A[] require alignment as well. That can only work when the struct is 16 bytes long. Or in other words, swapping the fields just moves the unused 7 bytes to the end of the struct.
Any struct in C# has an implied [StructLayout(LayoutKind.Sequential)] attribute. You would have to use LayoutKind.Explicit to force intentional mis-alignment of a field. Do beware the cost, misaligned fields can be quite expensive, slowing down reads and writes by as much as x3. Happens when a field straddles an L1 cache line boundary. And much, much more if the processor doesn't support misaligned accesses, not much of a problem today with the Itanium out of the picture. And you lose any atomicity guarantee you get from the CLR spec. You cannot intentionally misalign a field of a reference type (like string), that ruins the garbage collector.
It gets slightly more interesting if the struct has 3 or more members. Now you can indeed get ahead by ordering them. Like a byte, long, int, that would take 20 bytes in that order. If you re-arrange to byte, int, long than it takes 16 bytes, the int fits in the padding. If you override with LayoutKind.Auto then the CLR will do this for you.
I need to use this Kernel32 structure: NUMA_NODE_RELATIONSHIP
But the Reserved field is declared as: BYTE Reserved[20];
To declare the structure properly, is there a better way than declaring:
public struct NUMA_NODE_RELATIONSHIP
{
public UInt16 NodeNumber;
public BYTE Reserved1;
public BYTE Reserved2;
...
public BYTE Reserved20;
public GROUP_AFFINITY GroupMask;
}
or using:
[StructLayout(LayoutKind.Explicit)]
public struct NUMA_NODE_RELATIONSHIP2
{
[FieldOffset(0)]
public UInt16 NodeNumber;
// [FieldOffset(2)] // **
// public byte Reserved;
[FieldOffset(22)]
public GROUP_AFFINITY GroupMask;
}
** I'm not sure my code is ok... Here FieldOffset is a byte offset or multiple of OS word size (32 or 64 bits)? Does the WIN32 API pad align struct fields and if so, how? If both are padded or both are not padded, I'm fine, otherwise my code is bugged. That is not my main question. That insert is there only to mention that my code is not necessarily working as it is written. My main question is this one:
What is the best way to declare a pinvoke struct with an array in it?
I wonder if both will be fine, which one is preferred and/or if there is a better way of declaring that struct because both way I used add artifacts that make it long to write and/or harder to understand.
Declare it like this:
[StructLayout(LayoutKind.Sequential)]
public struct NUMA_NODE_RELATIONSHIP
{
public uint NodeNumber;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 20)]
public byte[] Reserved;
public GROUP_AFFINITY GroupMask;
}
There's no need for explicit layout. It makes more sense to let the marshaler lay out the struct using the same rules as the C++ compiler. Especially as GROUP_AFFINITY has alignment of 4 or 8 bytes depending on the machine pointer size. Obviously the above relies on you getting GROUP_AFFINITY right.
The WinAPI aligns most small types at their natural size, so WORD at 16 bits and DWORD at 32 bits. BYTE, being one byte, isn't aligned. Padding is inserted where needed to achieve the proper alignment for the next member.
What makes it tricky here is the aligment of GROUP_AFFINITY. I think it's 2 bytes because all its members are WORD.
The second reason I think that it's 2 byte aligned is because Microsoft tends to name its padding (!). If there was 2 bytes of unnamed padding after Reserved, Microsoft would have made it BYTE reserved[22] instead.
And yes, .Net field offsets are in bytes too.
Is there any convention over the algorithm used to make the layouts of structs on C?
I want to be able to have a code running in a vm to be able to have structures compatible with their C counterparts, just like C# interop works. For this I will need to know how the alignment algorithm works. I gather there must be a convetion for that, as it works nicely on C#. I have in mind the probable algorithm they have used to work this out, but I haven't found any proof it is the right one.
Here's how I think it works:
for each declared field (by order of declaration)
See if the field fits in the remaining bytes (until next alignment)
If it doesn't fit, align this field; otherwise add it to current offset
for example, on a 32-bit system for a struct like:
{
byte b1;
byte b2;
int32 i1;
byte b3;
}
would be like this with this algorithm:
{
byte b1;
byte b2;
byte[2] align1;
int32 i1;
byte b3;
byte[3] align2;
}
In general, structure alignment in C depends on the compiler used, and especially the compiler options in effect at the time the structure declaration is processed. You can't make any general assumptions except to say that for a particular structure in a particular program compiled with particular settings, the structure layout can be determined.
That said, your guess closely matches what most compilers are likely to do with default alignment settings.
What I mean is: Imagine we have a 8 byte variable that has a high value and low value. I can make one pointer point to the upper 4 bytes and other point to the lower 4 bytes, and set/retrieve their values without problems. Now, is there a way to get/set values for anything smaller than a byte? If instead of dividing it in two 4 bytes "variables", I'd want to consider eight 1 byte variables I could use a bool, but there is no defined smaller variable in c#. Would it possible to divide it to 16 just with pointers? Or even in 32, 64? It wouldn't right?
This is a pretty academic question, I know this can be achieved otherwise with bitshiffting, unions(Struct.Explicit), etc. Thanks!
No, C# does not support bit fields and a byte is the minimum amount of addressable memory. You can manually provide properties that change one or several specific bits but you have to provide packing/unpacking logic yourself:
public bool Bit5 {
get { return (field & 32) != 0; }
set { if (value) field |= 32; else field &= ~32; }
}
By the way, I don't know how you achieve it using LayoutKind.Explicit as the minimum FieldOffset you can specify is one byte.
As a side note, even C++ that can do this with bit fields will just hide the bitwise tricks and makes the compiler do it instead of you. There's no way you could grab something less than a byte from memory to a register, at least on x86 architecture.