Develop Reference

Serialization of object without structure

I am working on a software that will exchange data with a Siemens PLC (industrial controller).
In order for it to work, I need to be able to serialize and deserialize a byte array containing only the current values of the variables.
The problem I am facing is that Serialize/Deserialize methods add a lot of information beyond the current value of a variable.
The instance of the following class:
[Serializable]
public class VarMap
{
public byte var1;
public int64 var2;
public int32 var3;
}
After serialized, needed to be a byte array containing one value after the other, each ocuppying their size in bytes:
[var1 byte 1][var2 byte 1][var2 byte 2][var2 byte 3][var2 byte 4][var3 byte 1][var3 byte 2].
Any ideas how to make this happen dinamically according to the declaration of the class?

For complex reference types that refer to other reference types, there is not a simple way to flatten those out into a pure sequence of bytes and there is a reason serializers have to use more complex formats to handle such cases.
That said, the type you show is what is called a blittable type.
Blittable types only contain fields of other primitive types whose bit pattern can be directly copied.
By changing your class to a struct, you could generically persist such types to disk directly. This does require the use of an unsafe block, but the code is relatively simple.
There are ways to do the same thing without using unsafe code, but using unsafe is the most straight-forward since you can just take a pointer and use that to write directly to disk, the network, etc.
First, change to a struct:
[StructLayout(LayoutKind.Sequential)]
public struct VarMap
{
public byte var1;
public long var2;
public int var3;
}
Then, create one and write it directly to disk:
VarMap vm = new VarMap
{
var1 = 254,
var2 = 1234,
var3 = 5678
};
unsafe
{
VarMap* p = &vm;
int sz = sizeof(VarMap);
using FileStream fs = new FileStream(#"out.bin", FileMode.Create);
ReadOnlySpan<byte> span = new ReadOnlySpan<byte>(p, sz);
fs.Write(span);
}
However, if you look at the result in a hex viewer you will notice something:
24 bytes were written to disk, not the 13 that we might assume based on the types of the fields in your data structure.
This is because structures are padded in memory for alignment purposes.
Depending on what the device you are sending these bytes to actually expects (and endian-ness might also be an issue), this kind of technique may or may not work.
If you need absolute control, you probably want to hand-write the serialization code yourself rather than trying to find some generic mechanism to do this. It might be possible using reflection, but probably simpler to just convert your fields to the raw byte representation directly.

Marshalling a complex structure from C# to C

I'm reading all about structure marshalling between C and C# in 64bit environment without success.
What I need to do is to pass the structure
typedef struct DescStructTa
{
char* pszNa;
unsigned long ulTId;
char* pszT;
unsigned short usRId;
unsigned long ulOff;
unsigned long ulSi;
char szAcc[2];
unsigned char bySSize;
} DescStruct;
from a C# application to a C DLL not made by us calling the method
MT_DLL_DECL long GetAll(UINTPTR ulHandler, DescStruct **ppList, unsigned long *pulNumS);
After long search I wrote the following code:
[StructLayout(LayoutKind.Sequential, Pack = 8)]
public struct PlcSymbolDescStruct
{
[MarshalAs(UnmanagedType.LPStr)]
string pszNa;
UInt32 ulTId;
[MarshalAs(UnmanagedType.LPStr)]
string pszT;
[MarshalAs(UnmanagedType.U2)]
ushort usRId;
UInt32 ulOff;
UInt32 ulSi;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 2)]
string szAcc;
[MarshalAs(UnmanagedType.U1)]
byte bySSize;
}
[DllImport("DataSource.dll", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
[return: MarshalAs(UnmanagedType.U8)]public static extern long GetAll(ulong ulHandler, out DescStruct [] ppList,
out uint pulNumS);
Unfortunately this is still giving me a heap corruption.
If I try to pack less than 8 it gives (as expected) an access violation, but I expected this to work and I'm not able to figure out what could be wrong.
Any suggestion?
Thanks in advance!

I’m not sure you need Pack = 8. The default packing appears to be more or less compatible between C# and C++.
The szAcc field probably uses Unicode. To switch, specify CharSet=CharSet.Ansi in your [StructLayout]
Finally, if your C API has that double pointer to the array of them, you have to allocate that memory in a special way, either CoTaskMemAlloc or LocalAlloc, forgot which one.
It’s easier, also more efficient, to implement 2 functions instead. One to obtain the required size of the array. Another one to write them into caller-allocated buffer, a single pointer in C++ as opposed to double pointer, and UnmanagedType.LPArray in C#.
Update
allocation should not be necessary since the documentation states that the call returns a pointer to it's internal data structure
The runtime doesn’t know what was written in that documentation, attempts to free anyway, and crashes. You can use out IntPtr in C# API, and marshal manually.
If you have modern .NET, use unsafe, cast the IntPtr to PlcSymbolDescStruct* raw pointer, and construct ReadOnlySpan<PlcSymbolDescStruct> from that raw pointer.
if you’re stuck with old desktop .NET, call Marshal.PtrToStructure<PlcSymbolDescStruct> in a loop, incrementing the pointer by Marshal.SizeOf<PlcSymbolDescStruct>() bytes between loop iterations. No need to use unsafe code in this case.

Use Win32API: unions within structures generate TypeLoadException

I'd like to import an extern function of Win32API.
The Code from the API (in C) looks like this:
typedef struct _BLUETOOTH_ADDRESS {
union {
BTH_ADDR ullLong;
BYTE rgBytes[6];
};
} BLUETOOTH_ADDRESS;
My C# implementation looks like this:
[StructLayout(LayoutKind.Explicit, Size = 8)]
public struct BLUETOOTH_ADDRESS
{
[FieldOffset(0)]
public ulong ullLong;
[FieldOffset(2)]
public byte[] rgBytes;
};
The problem is: As soon as I create the struct, it throws me a TypeLoadException, error Code:
System.TypeLoadException: "Could not load type 'BLUETOOTH_ADDRESS' from assembly 'BleLab, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' because it contains an object field at offset 2 that is incorrectly aligned or overlapped by a non-object field."
Do you have any idea to fix this or where the problem is?
Best regards
Edit:
Forgot to provide the way of calling:
var ba = new Win32API.BLUETOOTH_ADDRESS();
ba.rgBytes = new byte[6];

There's really no point in trying to declare this as a union. It's an unsigned 64 bit type. Just use ulong instead of the struct.
If you never need to display the address, then you would need to pick out just the first 6 bytes of that ulong. A Bluetooth address is a 48 bit value, hence 6 bytes.
But for your purposes there's nothing to be gained by trying to express that nuance in the type used for interop. Which is why I would recommend working with ulong for the interop and picking out the meaningful bytes if necessary as a separate action.

To help you understand the error, note the following:
All members of a C union type have overlapping storage that begins at offset 0, so the offset of member rgBytes in your C# struct should use [FieldOffset(0)], not [FieldOffset(2)].
The type of the rgBytes member in the C union is a fixed array of 6 bytes. In your C# struct it is a byte[] array type. A (normal) array type in C# is a "reference type", which can be thought of as being like a C pointer. That is, the object just holds a reference (pointer) to a value on the heap.
You can create a fixed array using the fixed keyword as follows:
fixed byte rgBytes[6];
Fixed arrays are unsafe code, so the above needs to be declared unsafe. It is also declared public in your struct, so the full declaration of the rgBytes member can be as follows:
public unsafe fixed byte rgBytes[6];
Putting it all together gives the following C# declaration for BLUETOOTH_ADDRESS:
[StructLayout(LayoutKind.Explicit, Size = 8)]
public struct BLUETOOTH_ADDRESS
{
[FieldOffset(0)]
public ulong ullLong;
[FieldOffset(0)]
public unsafe fixed byte rgBytes[6];
};
You can probably leave out the Size = 8 part.
As per David Heffernan's answer, you are probably better off just using a ulong, especially as that will avoid any "unsafe" code.

How do I make fixed-size byte array user type in C#?

I'm converting an old Visual BASIC program to C#. It sends messages to some industrial machinery over ethernet. To do this it assembles a stream of bytes from fixed-size user defined chunks.
Most of these chunks are small and in C# it's easy to create structs of a few bytes or ints and control their size and layout using StructLayout's, for example
[StructLayout(LayoutKind.Sequential, Pack = 1)]
...so when we go into unmanaged space to do a bytewise copy we don't have byte order or padding problems.
But some of the VB6 structures are big arrays, for example,
Private Type SEND_MSG_BUFFER_320_BYTES
bytes(0 To 319) As Byte '320 bytes
End Type
and I'm struggling with how to do this in C#. I can make a fixed size array in a class, e.g.,
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class SOME_BYTES
{
public byte[] b = new byte[320];
}
but to do the byte-wise copy I need to be able to discover the size of this at runtime and System.Runtime.InteropServices.Marshal.SizeOf returns a 4 for this.
Any suggestions for how do do this will be much appreciated.

You can use fixed size buffers if you're okay with unsafe code, and changing your class into a struct:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public unsafe struct SomeBytes
{
public fixed byte MessageData[320];
}
Personally I would try to avoid all of this if possible. If you're just sending the data over the network, why do you need to "go into unmanaged space"? Can you remove that requirement somehow? (Maybe it's fundamental - but it's not clear from your question.)

You can use a fixed-size array:
unsafe struct SomeBytes {
public fixed byte b[320];
}

I think you want to do something like this:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class SOME_BYTES
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst=320)]
public byte[] b;
}
You would initialise it as follows:
SOME_BYTES data = new SOME_BYTES {b = new byte[320]};
Then you can populate data.b[] and use marshalling to get the data to send. The MarshalAs attribute tells the marshaller what fixed size buffer to use when marshalling the data.
You don't need to use the unsafe fixed keyword to do this kind of thing, and I strongly recommend that you avoid it.

Writing a C# Variable Length Structure to Binary and Reading it in in C++?

Okay, so i am continuing to work on my little game engine to teach me C#/C++ more. Now i am attempting to write a way of storing data in a binary format, that i created. (This is learning, i want to do this my self from scratch). What i am wondering what is the best way of dealing with variable length arrays inside a structure when reading it in C++?
E.g. Here is what i currently have for my structures:
[StructLayout(LayoutKind.Sequential)]
public struct FooBinaryHeader
{
public Int32 m_CheckSumLength;
public byte[] m_Checksum;
public Int32 m_NumberOfRecords;
public FooBinaryRecordHeader[] m_BinaryRecordHeaders;
public FooBinaryRecord[] m_BinaryRecords;
}
[StructLayout(LayoutKind.Sequential)]
public struct FooBinaryRecordHeader
{
public Int32 m_FileNameLength;
public char[] m_FileName;
public Int64 m_Offset;
}
[StructLayout(LayoutKind.Sequential)]
public struct FooBinaryRecord
{
public bool m_IsEncrypted;
public Int64 m_DataSize;
public byte[] m_Data;
}
Now how would i go about in C++ to actually read this in as a structure in C++? I was kinda hoping to get around reading each of the elements one by one and copying them into a structure.
The only real tutorial i found on this is this: http://www.gamedev.net/community/forums/topic.asp?topic_id=310409&whichpage=1&#1989200
I'll take a wild guess and say reading this into a C++ structure is not really possible correct?

There's no such thing as a variable length array in a structure.
Suppose I had a structure point such as
struct point
{
int x;
int y;
}
If I wanted an array of 5 of these, the compiler would essentially reserve space for 10 ints. What happens if I ask for an array of structures, of which each contains a variable length array? There's no way to align those in memory since we can't know how much space to reserve for each one.
What you can do is to declare a pointer to the type of which you want a variable length array, because a pointer is a constant size. Then, you allocate enough memory for however many instances of that type, and point to it that way. You'll probably need to also add a length field to the struct so you know exactly how far you can go past the pointer before you risk segfaulting.
It might get a little hairy going back and forth between managed and unmanaged code and allocating and freeing memory, but that's another good exercise for learning C++ and C# together, if anything.

You can read it from binary format mapping a copy of these structures. Each array should be treated as a pointer and you should have a integer with size of this array.
For example in
C#
[StructLayout(LayoutKind.Sequential)]
public struct A
{
public Int32 m_CheckSumLength;
public byte[] m_Checksum;
}
C++
struct A {
int length
char* vector
}
Notes: byte has the same size of char.
When you read from a binary you can read the first 4 byte (int is 32 aka 4 byte) and allocate 4 + (readed length) after that you can read directly to the allocated buffer and treat as a A structure.

Use Marshall.StructToPtr and copy length of structure.