Want to do this:
(EDIT: bad sample code, ignore and skip below)
struct RECORD {
char[] name = new char[16];
int dt1;
}
struct BLOCK {
char[] version = new char[4];
int field1;
int field2;
RECORD[] records = new RECORD[15];
char[] filler1 = new char[24];
}
But being unable to declare array sizes in struct, how do I reconfigure this?
EDIT: The reason for the layout is I'm using BinaryReader to read a file written with C structs. Using BinaryReader, and a C# struct union (FieldOffset(0)), I'm wanting to load the header as a byte array, then read it as it was intended originally.
[StructLayout(LayoutKind.Sequential)]
unsafe struct headerLayout
{
[FieldOffset(0)]
char[] version = new char[4];
int fileOsn;
int fileDsn;
// and other fields, some with arrays of simple types
}
[StructLayout(LayoutKind.Explicit)]
struct headerUnion // 2048 bytes in header
{
[FieldOffset(0)]
public byte[] headerBytes; // for BinaryReader
[FieldOffset(0)]
public headerLayout header; // for field recognition
}
Use fixed size buffers:
[StructLayout(LayoutKind.Explicit)]
unsafe struct headerUnion // 2048 bytes in header
{
[FieldOffset(0)]
public fixed byte headerBytes[2048];
[FieldOffset(0)]
public headerLayout header;
}
Alternativ you can just use the struct and read it with the following extension method:
private static T ReadStruct<T>(this BinaryReader reader)
where T : struct
{
Byte[] buffer = new Byte[Marshal.SizeOf(typeof(T))];
reader.Read(buffer, 0, buffer.Length);
GCHandle handle = default(GCHandle);
try
{
handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
return (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
}
finally
{
if (handle.IsAllocated)
handle.Free();
}
}
Unmanaged structures can contain embedded arrays. By default, these embedded array fields are marshaled as a SAFEARRAY. In the following example, s1 is an embedded array that is allocated directly within the structure itself.
Unmanaged representation
struct MyStruct {
short s1[128];
}
Arrays can be marshaled as UnmanagedType.ByValArray, which requires you to set the MarshalAsAttribute.SizeConst field. The size can be set only as a constant. The following code shows the corresponding managed definition of MyStruct.
C#VB
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct {
[MarshalAs(UnmanagedType.ByValArray, SizeConst=128)] public short[] s1;
}
I wouldn't use that pattern in the first place. This kind of memory mapping may be appropriate in c, but not in a high level language like C#.
I'd just write a call to the binary reader for each member I want to read. This means you can use classes and write them in a clean high level way.
It also takes care of endian issues. Whereas memory mapping will break when used on different endian systems.
Related question: Casting a byte array to a managed structure
So your code would look similar to the following (add access modifiers etc.):
class Record
{
char[] name;
int dt1;
}
class Block {
char[] version;
int field1;
int field2;
RECORD[] records;
char[] filler1;
}
class MyReader
{
BinaryReader Reader;
Block ReadBlock()
{
Block block=new Block();
block.version=Reader.ReadChars(4);
block.field1=Reader.ReadInt32();
block.field2=Reader.ReadInt32();
block.records=new Record[15];
for(int i=0;i<block.records.Length;i++)
block.records[i]=ReadRecord();
block.filler1=Reader.ReadChars(24);
return block;
}
Record ReadRecord()
{
...
}
public MyReader(BinaryReader reader)
{
Reader=reader;
}
}
Using unsafe code and fixed size buffer this can be done: http://msdn.microsoft.com/en-us/library/zycewsya.aspx
Fixed size buffers are inline-bytes of the struct. They don't live inside of a separate array like your char[] does.
Unless you really need a struct, you can do this with a class. A class is basically a struct, and will be used exactly the same way, but it can contain methods inside. One of these methods is the constructor, which will initialize default values inside it once you create a new instance with "new". To create a constructor, put a method with the same name of the class inside it. It may receive arguments if you wish.
class RECORD
{
public int dt1;
public char[] name;
public RECORD => name = new char[16] // if it is one-line the {} can be =>
}
class BLOCK
{
public char[] version;
public int field1;
public int field2;
public RECORD[] records;
public char[] filler1;
public BLOCK()
{
records = new RECORD[15];
filler1 = new char[24];
version = new char[4];
}
}
This way when you create a new item of type BLOCK, it will be pre-initialized:
var myblock = new BLOCK();
Console.WriteLine(myblock.records.Length); // returns 15
Console.WriteLine(myblock.records[0].Length); // returns 16
Console.WriteLine(myblock.filler1.Length); // returns 24
Related
I would like to get the offset of a field in an unmanaged structure. For this I use the Marshal.OffsetOf method and I realized that the result does not reflect the Packing used with StructLayout
Take this example:
using System;
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Sequential, Pack = 4)]
unsafe public struct NETGROUP
{
public bool Advise;
public bool Active;
public int UpdateRate;
public double DeadBand;
}
namespace MyApp // Note: actual namespace depends on the project name.
{
internal class Program
{
static void Main(string[] args)
{
unsafe
{
// Technique 1 (not correct)
var oA = Marshal.OffsetOf(typeof(NETGROUP), "Advise").ToInt32();//0
var oB = Marshal.OffsetOf(typeof(NETGROUP), "Active").ToInt32();//4
var oC = Marshal.OffsetOf(typeof(NETGROUP), "UpdateRate").ToInt32();//8
var oD = Marshal.OffsetOf(typeof(NETGROUP), "DeadBand").ToInt32();//12
// Technique 2 (correct)
NETGROUP ex = new NETGROUP();
byte* addr = (byte*)&ex;
var oAa = (byte*)(&ex.Advise) - addr;//0
var oBb = (byte*)(&ex.Active) - addr;//1
var oCc = (byte*)(&ex.UpdateRate) - addr;//4
var oDd = (byte*)(&ex.DeadBand) - addr;//8
}
}
}
}
I need to retrieve this offset in a generic constructor but the second technique (which is the correct one) does not allow me to achieve it without specifying the type explicitly
public CMember(Type type, string pszName, CType pType, uint uMod = Modifier.TMOD_NON, int nDim = 0) : base(DefineConstants.TOKN_MBR)
{
m_sName = pszName;
m_nOffset = Marshal.OffsetOf(type, pszName).ToInt32(); // !!!
m_pType = pType;
m_uMod = uMod;
m_nDim = nDim;
}
Do you have an idea ?
The OffsetOf only returns the layout of the unmanaged instances of a struct
OffsetOf provides the offset in terms of the unmanaged structure layout, which does not necessarily correspond to the offset of the managed structure layout. Marshaling the structure can transform the layout and alter the offset.
See also StructLayout
The common language runtime controls the physical layout of the data fields of a class or structure in managed memory. However, if you want to pass the type to unmanaged code, you can use the StructLayoutAttribute attribute to control the unmanaged layout of the type.
'ex' is a managed instance so you get the default layout
Marshal.OffsetOf is working as expected, the "issue" is the System.Boolean which is a non-blittable type and has unmanaged size of 4:
Console.WriteLine(sizeof(bool)); // 1
Console.WriteLine(Marshal.SizeOf(typeof(bool)));// prints 4
From UnmanagedType enum docs:
Bool A 4-byte Boolean value (true != 0, false = 0). This is the Win32 BOOL type
Changing struct to contain byte fields instead of bool ones produces the expected output:
[StructLayout(LayoutKind.Sequential, Pack = 4)]
unsafe public struct NETGROUP
{
public byte Advise;
public byte Active;
...
}
Console.WriteLine(Marshal.OffsetOf(typeof(NETGROUP), "Advise")); // 0
Console.WriteLine(Marshal.OffsetOf(typeof(NETGROUP), "Active")); // 1
Another approach is to marshal bool as UnmanagedType.I1:
A 1-byte signed integer. You can use this member to transform a Boolean value into a 1-byte, C-style bool (true = 1, false = 0).
[StructLayout(LayoutKind.Sequential, Pack = 4)]
public struct NETGROUP
{
[MarshalAs(UnmanagedType.I1)]
public byte Advise;
[MarshalAs(UnmanagedType.I1)]
public byte Active;
...
}
Some more info here.
I am trying to allocate an array of structures in C#. For example,
public struct Channel {
int ChannelId;
// other stuff goes here...
}
public struct FrameTraffic {
public int FrameId;
public int MaxChannels;
public Channel[] Channels;
public FrameTraffic(int dummyCS0568 = 0)
{
this.FrameId = 0;
MaxChannels = TableMgr.MaxChannels;
Channels = new Channel[TableMgr.MaxChannels];
}
}
But when I go to allocate an array of FrameTraffic structures, I see that Channels is null. This tells me that Channels is a reference rather than an array of structures. Am I correct? If so, then allocating the Channels array shouldn't embed the array into the structure, but simply satisfy the reference in the structure. I want the structures embedded. Is there a way to do this? Or am I incorrect in my assumptions?
Answering the later part of your question and disregarding any other problem. Yes you are correct, this will be a reference to an array. However, if you wanted to embed the array in the struct you can use a fixed sized buffer using the fixed and unsafe keywords. However that can only be known at design time, also it can only be of the following value types and not a user defined struct.
bool, byte, char, short, int, long, sbyte, ushort, uint, ulong, float, or double.
So in short, what you want to do is not possible, you may need to clarify why you need this or re-think your problem
You need to use the correct marshalling attribute, and it needs to have a fixed size, say 40
public struct FrameTraffic
{
public int FrameId;
public int MaxChannels;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 40)]
public Channel[] Channels;
}
I was able to replicate the null issue not sure if its the same with yours :
There are two things that i think is possibly causing this :
You are just initializing the array with size but not assigning any values
You might be initializing FrameTraffic with default construct instead of what you have defined (this caused the actual NPE for me)
Below is how you can adjust your code: (I have hardcoded values which is brought by TableMgr.MaxChannels since i dont have that)
class Program
{
static void Main()
{
FrameTraffic fT = new FrameTraffic(0);
foreach (var item in fT.Channels)
{
Console.WriteLine(item.ChannelId);
}
Console.Read();
}
}
public struct Channel
{
public int ChannelId; //missing public exposer if you really want to reassign
// other stuff goes here...
}
public struct FrameTraffic
{
public int FrameId;
public Channel[] Channels;
public FrameTraffic(int dummyCS0568 = 0)
{
this.FrameId = 0;
const int MaxChannels = 1;
//array requires size and its values assigned here
Channels = new Channel[MaxChannels]{ new Channel { ChannelId = 1 } };
}
}
I'm trying to have a stack allocated array inside a struct. Well the pointer I mean. But I'd like the allocation to be done without extra code because I know the size when I write the code (I don't want to do a bunch of new when I create my struct).
If I can even do it without unsafe context that's perfect.
I tried some stuff, but it's not doing fine. I'm brand new to C# so there is probably a way to do it that I didn't see !
public struct TestValue {int value; }
[StructLayout(LayoutKind.Sequential)]
public struct TestArray {
[MarshalAs(UnmanagedType.ByValArray, SizeConst=128)] public TestValue[] s1;
}
public struct TestSpan
{
Span<TestValue> data= stackalloc TestValue[10];
}
using System.Runtime.InteropServices;
public struct TestValue {int value; }
[StructLayout(LayoutKind.Sequential)]
public struct TestArray {
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst=128)] public TestValue[] s1;
}
public class Foo
{
void test()
{
TestArray test = new TestArray();
test.s1[10] = new TestValue();
}
}
I needed just a small change in the end!
There isn't much explaining to this, this is what I have:
public struct PACKET_HEADER
{
public string computerIp;
public string computerName;
public string computerCustomName;
};
public struct PACKET
{
public PACKET_HEADER pktHdr;
public PACKET_DATA pktData;
};
public struct PACKET_DATA
{
public Command command;
public string data;
};
public struct DATA_MESSAGE
{
public string message;
};
public struct DATA_FILE
{
public string fileName;
public long fileSize;
};
Basically I want the data field in PACKET_DATA to be able to be either DATA_FILE or DATA_MESSAGE. I know the type needs to be changed but I don't know what to, is generics an option?
the end result should be so that I can do either:
pktData.data.fileName
or
pktData.data.message
EDIT
i could do:
public struct PACKET_DATA
{
public Command command;
public string data;
public DATA_MESSAGE data_message;
public DATA_FILE data_file;
};
and just set the data_message or file to null when ever i don't need them? how would this impact the serialization / byte array and the data being sent. If I used classes would I not have the same problem
EDIT 2
public struct PACKET_MESSAGE
{
public PACKET_HEADER pktHdr;
public Command command;
public DATA_MESSAGE pktData;
};
public struct PACKET_FILE
{
public PACKET_HEADER pktHdr;
public Command command;
public DATA_FILE pktData;
};
Edit 3
I have a sterilizer and de-sterilizer that works with my original example, if there are no hiccups with that then the actual serialization is done.
EDIT 4
everything seems to be working, apart from one things my serializer is getting "Attempted to read or write protected memory. This is often an indication that other memory is corrupt." gunna have a look at it when post my working solution :)
EDIT 5
public static byte[] Serialize(object anything)
{
int rawsize = Marshal.SizeOf(anything);
byte[] rawdatas = new byte[rawsize];
GCHandle handle = GCHandle.Alloc(rawdatas, GCHandleType.Pinned);
IntPtr buffer = handle.AddrOfPinnedObject();
Marshal.StructureToPtr(anything, buffer, false);
handle.Free();
return rawdatas;
}
public static object Deserialize(byte[] rawdatas, Type anytype)
{
int rawsize = Marshal.SizeOf(anytype);
if (rawsize > rawdatas.Length)
return null;
GCHandle handle = GCHandle.Alloc(rawdatas, GCHandleType.Pinned);
IntPtr buffer = handle.AddrOfPinnedObject();
object retobj = Marshal.PtrToStructure(buffer, anytype);
handle.Free();
return retobj;
}
FINAL
The structs:
public struct PACKET_HEADER
{
public string computerIp;
public string computerName;
public string computerCustomName;
};
public struct PACKET
{
public PACKET_HEADER pktHdr;
public PACKET_DATA pktData;
};
public struct PACKET_DATA
{
public Command command;
public IDATA data;
public T GetData<T>() where T : IDATA
{
return (T)(data);
}
}
public interface IDATA { }
public struct DATA_MESSAGE : IDATA
{
public string message;
}
public struct DATA_FILE : IDATA
{
public string fileName;
public long fileSize;
}
How to create a new Packet (probally could combine together tbh):
public static PACKET CreatePacket(Command command)
{
PACKET packet;
packet.pktHdr.computerIp = Settings.ComputerIP;
packet.pktHdr.computerName = Settings.ComputerName;
packet.pktHdr.computerCustomName = Settings.ComputerCustomName;
packet.pktData.command = command;
packet.pktData.data = null;
return packet;
}
public static PACKET CreatePacket(Command command, DATA_MESSAGE data_message)
{
PACKET packet;
packet.pktHdr.computerIp = Settings.ComputerIP;
packet.pktHdr.computerName = Settings.ComputerName;
packet.pktHdr.computerCustomName = Settings.ComputerCustomName;
packet.pktData.command = command;
packet.pktData.data = data_message;
return packet;
}
public static PACKET CreatePacket(Command command, DATA_FILE data_file)
{
PACKET packet;
packet.pktHdr.computerIp = Settings.ComputerIP;
packet.pktHdr.computerName = Settings.ComputerName;
packet.pktHdr.computerCustomName = Settings.ComputerCustomName;
packet.pktData.command = command;
packet.pktData.data = data_file;
return packet;
}
(de) serialization above.
Simple example:
PACKET packet = Packet.CreatePacket(command, data_file);
byte[] byData = Packet.Serialize(packet);
other end:
PACKET returnPacket = (PACKET)Packet.Deserialize(socketData.dataBuffer, typeof(PACKET));
// Get file
string fileName = returnPacket.pktData.GetData<DATA_FILE>().fileName;
long fileSize = returnPacket.pktData.GetData<DATA_FILE>().fileSize;
All seems to be working nice and dandy :)
This question needs a clear answer, so I'll try to sum up:
If you want to take a C# data structure and convert it into a byte array, you can do it with structs and Marshaling, or with classes (or structs, but why would you) and a serialization framework (like BinaryFormatter), or custom serialization logic (like with BinaryWriter). We could have a debate about which is better, but lets assume for now that we're going with structs, and we're using Marshaling. Although I will say, that structs are very limited, and should be used mainly as necessary for interop with Win32 API functions.
So the issue is, we have a container struct, which may contain one of two types of child structs. If you're going to Marshal a struct, things like generics and or using a common Interface for your child struct types are not going to fly. Basically you're only option is to have the container have both structs and a bool flag indicating which of the structures is to be used. This has the downside of increasing the size of your packets because you're sending the unused child struct as well.
In the case at hand, the result looks like this:
public struct PACKET_DATA
{
public Command command;
public string data;
public bool is_message_packet;
public DATA_MESSAGE data_message;
public DATA_FILE data_file;
};
That said, in your case using structs and Marshalling is only really going to work within your own process, because your structs contain Strings. When a struct contains pointers to non-fixedlength strings, those strings are allocated elsewhere and will not be part of the byte array you copy, only pointers to them will be. You also need to call Marshal.DestroyStructure at some point, with the IntPtr you passed to StructureToPtr, in order to clean up these string resources.
So the moral of the story: can you make structs that do what you asked initially: yes. Should you be using them like you are: no. Because you have a variable size data structure that you are trying to send over a network (I presume since the struct is called PACKET), structs are not going to work, you really need to use some kind of serialization framework or custom serialization logic.
public struct PACKET_DATA
{
public IData data;
public T GetData<T>() where T : IDATA
{
return (T)data;
}
}
public interface IDATA { }
public struct DATA_MESSAGE : IDATA
{
public string message;
}
public struct DATA_FILE : IDATA
{
public string fileName;
public long fileSize;
}
PACKET_DATA packetData = new PACKET_DATA();
packetData.data = new DATA_MESSAGE();
var message = packetData.GetData<DATA_MESSAGE>().message;
You can do it with generics, but then a type parameter will propagate to the PACKET struct which I'm guessing will make it awkward to work with and not be what you want.
What's the purpose of using structs here, rather than classes? Is it for interop? (In which case, the interop scenario will dictate the right solution). Or is it to avoid boxing / heap allocation?
How about defining a bogus interface that both structs inherit from? Of course this will not solve your serialization issues, but as was said, you'd probably need a custom serialization method anyway.
public interface IDataType
{
}
public struct PACKET_DATA
{
public Command command;
public IDataType data;
};
public struct DATA_MESSAGE : IDataType
{
public string message;
};
public struct DATA_FILE : IDataType
{
public string fileName;
public long fileSize;
};
I'm working with packets that I can receive or send as a Byte array,
that has a fixed structure. So I'm trying to create an effective
union as follows:
using System; // etc..
namespace WindowsApplication1
{
public partial class Main : Form
{
public const int PktMaxSize = 124;
// ...etc..
// ...will use Pkt structure below...
}
[System.Runtime.InteropServices.StructLayout(LayoutKind.Explicit)]
public struct Pkt
{
[System.Runtime.InteropServices.FieldOffset(0)]
fixed Byte bytes[Main.PktMaxSize]; // complete byte pkt
[System.Runtime.InteropServices.FieldOffset(0)]
fixed Byte PktID[8];
[System.Runtime.InteropServices.FieldOffset(8)]
UInt16 Properties;
// ...etc..
}
}
I get a C# error
Pointers and size buffers may only be used in an unsafe context
What do I need to do to be able to create and use "unsafe" structures in a safe env?
Thanks for your help - open to any suggestions as to how to handle packet structures that can easily be converted to and from a fixed byte stream being received by (or sent by) a C++ interop class.
The use of the fixed keyword requires Pkt and all methods using it to be declared unsafe, e.g.,
[StructLayout(LayoutKind.Explicit)]
public unsafe struct Pkt
{
[FieldOffset(0)]
fixed Byte bytes[124];
...
}
If you don't want to use unsafe code you can declare Pkt as follows:
[StructLayout(LayoutKind.Explicit)]
public struct Pkt
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 124)]
[FieldOffset(0)]
Byte[] bytes;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)]
[FieldOffset(0)]
Byte[] PktID;
[FieldOffset(8)]
UInt16 Properties;
}
Use the unsafe keyword on the method or block:
unsafe static void DoSomethingUnsafe()
{
// use Pkt structure
}
static void DoSomething()
{
// do safe things
unsafe
{
// use Pkt structure
}
}
You must also enable unsafe code via the /unsafe option or the Project > Properties > Build tab in Visual Studio.