Develop Reference

Marshal.Sizeof() returning unexpected value

I'm debugging code in C# written by a 3rd party. The project is an old C++ project that was rewritten in C# by a contractor, and I have no access to the contractor. I authored the original C++ version.
The issue is when the C# code gets the size of a structure that represents data received over a UDP connection.
The struct is defined as:
[StructLayout(LayoutKind.Sequential,Pack=1)]
internal class PROXY_HDR {
public ushort pad;
public ushort label;
public char flags;
public ushort length;
public char[] ip = new char[4];
public ushort port;
}
The size of this struct is retrieved as:
int size = Marshal.Sizeof(typeof(PROXY_HDR));
and the value returned is 17 instead of the expected 13. With a 4-byte difference, I suspect the ip member, but only because it's expressed differently than the other members (with 'new'), but I have no other basis to decide.
I don't typically use this type of marshaling in my C# code that parses received packets without a problem, so I don't know how to modify this struct definition to make it 'line up' with the original version size-wise.
I could replace the Marshal line with
int size = 13;
but that's cheating, right?
Can I modify this layout somehow to get the size to come out right?

Add this to the structure:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
internal class PROXY_HDR
{
public ushort pad;
public ushort label;
public byte flags;
public ushort length;
[MarshalAs(UnmanagedType.ByValArray,
SizeConst = 4, ArraySubType = UnmanagedType.U1)]
public byte[] ip;
public ushort port;
}
This will tell the compiler to treat it as a typical C-style array, not a pointer. This should be byte, as an IP address is an unsigned char array. char typically isn't used in these types of headers, it's usually byte (i.e: unsigned char)

Using C#'s Marshal.PtrToStructure on a C++ struct containing two bitsets

I'm trying to use my existing code to map a C++ structure containing std::bitsets to a C# version of it.
My code works great on these simple objects:
// C++ struct
struct log_t
{
uint16 current;
bool complete[64];
};
C# struct
[StructLayout(LayoutKind.Sequential)]
public struct Log
{
public UInt16 current;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 64, ArraySubType = UnmanagedType.Struct)]
public byte[] complete;
}
// Convert the C# struct back to bytes for the C++ app to use
private byte[] GetObjectBytes(object obj)
{
int size = Marshal.SizeOf(obj);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(obj, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
How do I map the following C++ structs to C# structs that contains two std::bitsets? I wish to use them in the same manner as above.
struct items_table
{
std::bitset<512> keyList;
std::bitset<512> seenList;
};
struct items
{
std::array<items_table, 7> tables;
};
SOLUTION
I used byte[] arrays in C# to represent the bitsets in C++. I then had to access each bit like this:
private void AddItem(int ItemId, KeyItems keyitems)
{
decimal byteIndex = (ItemId % 512) / 8;
decimal bitInByteIndex = (ItemId % 512) % 8;
keyitems.tables[ItemId / 512].keyList[(int)byteIndex] |= (byte)(1 << (byte)bitInByteIndex);
}

We use interop a lot where I work, and though I haven't had to handle this situation, I suggest you treat your bitsets as byte arrays (byte[] in C#, I believe char[] in C++ unless you're aliasing char). After all 512 bits is just 64 contiguous bytes.
I admit I haven't given you a full explanation of the syntax, my apologies, but I hope this gets you pointed in the right direction.
If your C# end needs to use them as BitArray instances, you could create properties on your struct there and just make them calculated, based on your byte[]s. That does no harm for interop; the marshaller doesn't try to convert them across code boundaries in either direction.

Strange unmarshalling behavior with union in C#

I want to export a C-like union into a byte array, like this :
[StructLayout(LayoutKind.Explicit)]
struct my_struct
{
[FieldOffset(0)]
public UInt32 my_uint;
[FieldOffset(0)]
public bool other_field;
}
public static void Main()
{
var test = new my_struct { my_uint = 0xDEADBEEF };
byte[] data = new byte[Marshal.SizeOf(test)];
IntPtr buffer = Marshal.AllocHGlobal(data.Length);
Marshal.StructureToPtr(test, buffer, false);
Marshal.Copy(buffer, data, 0, data.Length);
Marshal.FreeHGlobal(buffer);
foreach (byte b in data)
{
Console.Write("{0:X2} ", b);
}
Console.WriteLine();
}
The output we get (https://dotnetfiddle.net/gb1wRf) is 01 00 00 00 instead of the expected EF BE AD DE.
Now, what do we get if we change the other_field type to byte (for instance) ?
Oddly, we get the output we wanted in the first place, EF BE AD DE (https://dotnetfiddle.net/DnXyMP)
Moreover, if we swap the original two fields, we again get the same output we wanted (https://dotnetfiddle.net/ziSQ5W)
Why is this happening? Why would the order of the fields matter ? Is there a better (reliable) solution for doing the same thing ?

This is an inevitable side-effect of the way a structure is marshaled. Starting point is that the structure value is not blittable, a side-effect of it containing a bool. Which takes 1 byte of storage in the managed struct but 4 bytes in the marshaled struct (UnmanagedType.Bool).
So the struct value cannot just be copied in one fell swoop, the marshaller needs to convert each individual member. So the my_uint is first, producing 4 bytes. The other_field is next, also producing 4 bytes at the exact same address. Which overwrites everything that my_uint produced.
The bool type is an oddity in general, it never produces a blittable struct. Not even when you apply [MarshalAs(UnmanagedType.U1)]. Which in itself has an interesting effect on your test, you'll now see that the 3 upper bytes produced by my_int are preserved. But the result is still junk since the members are still converted one-by-one, now producing a single byte of value 0x01 at offset 0.
You can easily get what you want by declaring it as a byte instead, now the struct is blittable:
[StructLayout(LayoutKind.Explicit)]
struct my_struct {
[FieldOffset(0)]
public UInt32 my_uint;
[FieldOffset(0)]
private byte _other_field;
public bool other_field {
get { return _other_field != 0; }
set { _other_field = (byte)(value ? 1 : 0); }
}
}

I admit that I don't have an authoritative answer for why Marshal.StructureToPtr() behaves this way, other than that clear it is doing more than just copying bytes. Rather, it must be interpreting the struct itself, marshaling each field individually to the destination via the normal rules for interpreting that field. Since bool is defined to only ever be one of two values, the non-zero value gets mapped to true, which marshals to raw bytes as 0x00000001.
Note that if you really just want the raw bytes from the struct value, you can do the copying yourself instead of going through the Marshal class. For example:
var test = new my_struct { my_uint = 0xDEADBEEF };
byte[] data = new byte[Marshal.SizeOf(test)];
unsafe
{
byte* pb = (byte*)&test;
for (int i = 0; i < data.Length; i++)
{
data[i] = pb[i];
}
}
Console.WriteLine(string.Join(" ", data.Select(b => b.ToString("X2"))));
Of course, for that to work you will need to enable unsafe code for your project. You can either do that for the project in question, or build the above into a separate helper assembly where unsafe is less risky (i.e. where you don't mind enabling it for other code, and/or don't care about the assembly being verifiable, etc.).

Is there a generic way to write a struct to bytes in Big Endian format?

I've found questions such as this one, which have come close to solving my dilemma. However, I've yet to find a clean approach to solving this issue in a generic manner.
I have a project that has a lot of structs that will be used for binary data transmission. This data needs to be Big Endian and, of course, most .Net architecture is Little Endian. This means that when I convert my structs to bytes, the byte order for my values are reversed.
Is there a fairly straight-forward approach to either forcing my structs to contain data in Big Endian format, or is there a way to generically write these structs to byte arrays (and byte arrays to structs) that output Big Endian data?
Here is some sample code for what I've already done.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public unsafe struct StructType_1
{
short shortVal;
ulong ulongVal;
int intVal;
}
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public unsafe struct StructType_2
{
long longVal_1;
long longVal_2;
long longVal;
int intVal;
}
...
public static class StructHelper
{
//How can I change the following methods so that the struct data
//is converted to and from BigEndian data?
public static byte[] StructToByteArray<T>(T structVal) where T : struct
{
int size = Marshal.SizeOf(structVal);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(structVal, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
public static T StructFromByteArray<T>(byte[] bytes) where T : struct
{
int sz = Marshal.SizeOf(typeof(T));
IntPtr buff = Marshal.AllocHGlobal(sz);
Marshal.Copy(bytes, 0, buff, sz);
T ret = (T)Marshal.PtrToStructure(buff, typeof(T));
Marshal.FreeHGlobal(buff);
return ret;
}
}

If you don't mind reading and writing each field to a stream (which may have performance implications) you could use Jon Skeet's EndianBinaryWriter: https://stackoverflow.com/a/1540269/106159
The code would look something like this:
public unsafe struct StructType_2
{
long longVal_1;
long longVal_2;
long longVal;
int intVal;
}
using (MemoryStream memory = new MemoryStream())
{
using (EndianBinaryWriter writer = new EndianBinaryWriter(EndianBitConverter.Big, stream))
{
writer.Write(longVal_1);
writer.Write(longVal_2);
writer.Write(longVal);
writer.Write(intVal);
}
byte[] buffer = memory.ToArray();
// Use buffer
}
You would use the EndianBinaryReader for data going in the opposite direction.
This does of course have two fairly large drawbacks:
It's fiddly writing code to read and write each field.
It may be too slow to do it this way, depending on performance requirements.
Also have a look at this answers to this similar question: Marshalling a big-endian byte collection into a struct in order to pull out values

Given the example on the BitConverter that shows converting a uint, I would suspect not.

Best way to represent Bit Arrays in C#?

I am currently building a DHCPMessage class in c#.
RFC is available here : http://www.faqs.org/rfcs/rfc2131.html
Pseudo
public object DHCPMessage
{
bool[8] op;
bool[8] htype;
bool[8] hlen;
bool[8] hops;
bool[32] xid;
bool[16] secs;
bool[16] flags;
bool[32] ciaddr;
bool[32] yiaddr;
bool[32] siaddr;
bool[32] giaddr;
bool[128] chaddr;
bool[512] sname;
bool[1024] file;
bool[] options;
}
If we imagine that each field is a fixed length bit array, what is :
The most versitile
Best practice
way of representing this as a class???
OR.. how would you write this? :)

For starters, you might try the BitArray class. No need to reinvent the wheel here.
If you're worried about it taking up too much space/memory, don't be. Just initialize it to the right size:
BitArray op = new BitArray(8);
(The above will hold 8 bits and should take up 1 byte)

You are on the wrong track with this, it isn't a bit vector. The message is defined in "octets", better known as "bytes". An equivalent C# declaration that you can use with Marshal.PtrToStructure is:
[StructLayout(LayoutKind.Sequential, Pack=1, CharSet=CharSet.Ansi)]
struct DHCPMessage {
public byte op;
public byte htype;
public byte hlen;
public byte hops;
public uint xid;
public ushort secs;
public ushort flags;
public uint ciaddr;
public uint yiaddr;
public uint siaddr;
public uint giaddr;
[MarshalAs(UnmanagedType.ByValArray, SizeConst=16)]
public byte[] chaddr;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst=64)]
public string sname;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst=128)]
public string file;
}
You'll need to handle the variable length options field separately.

Are you sure you want to be using bit arrays for some of these? For example, you can use byte for 8 bits, int for 32 bits, and byte arrays for pieces that map to null terminated strings like 'sname' for example. Then you can use simple bitwise operators (&, |) to check/manipulate the bits.
Here are some posts I did on converting TCP header to a structure, which also covers endianness etc.
http://taylorza.blogspot.com/2010/04/archive-structure-from-binary-data.html
http://taylorza.blogspot.com/2010/04/archive-binary-data-from-structure.html
These are quite old, I migrated them from my old blog just so they do not get lost.