I have a C# app which read/write messages of some predefined protocol from/to USB. So I have at least 2 options here. One is serialize/Deserialize. The other is marshal. At the beginning I was picking the first option and it worked fine. But after more and more message types are introduced I am annoyed by the cumbersome implementation for each message type and feel the marshal may be a better way to go. I haven't use marshal before. Is it the right way to go? I did some test. One problem I have is when I have a structure with an array, how to write to it? Such as the following:-
[StructLayout(LayoutKind.Sequential, Size=TotalBytesInStruct),Serializable]
public struct SomeData
{
/// char[15]
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 15)]
public sbyte[] data;
/// int[15]
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 15)]
public int[] prob;
}
It appears that the data and prob are both reference and I have to new a object to use them, which sounds not quite right to me. I need the whole struct to be a continuous block of memory, which I don't think the new operator will do that for me.
Any suggestion is really appreciated
The following is how I did by serialization.
List<byte> rawData = new List<byte>();
rawData.AddRange(BitConverter.GetBytes(ProtocolVersion));
// 16 bytes for operator ID
byte[] temp = new byte[16];
Array.Copy(Encoding.ASCII.GetBytes(OperatorId), temp, OperatorId.Length > temp.Length ? temp.Length : OperatorId.Length);
rawData.AddRange(temp);
// 16 bytes for operator password
Array.Clear(temp, 0, temp.Length);
Array.Copy(Encoding.ASCII.GetBytes(OperatorPassword), temp, OperatorPassword.Length > temp.Length ? temp.Length : OperatorPassword.Length);
rawData.AddRange(temp);
The following is how I did by marshaling
static byte[] RawSerialize<T>(T obj)
{
if (obj == null)
{
return null;
}
int size = Marshal.SizeOf(obj);
byte[] result = new byte[size];
IntPtr buffer = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(obj, buffer, false);
Marshal.Copy(buffer, result, 0, size);
return result;
}
It seems that I only need to new an array like the following and it works fine.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class SomeData
{
/// char[15]
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 15)]
public byte[] data = new byte[15];
/// int[15]
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 15)]
public int[] prob = new int[15];
}
Related
I want to use my laptop to communicate with MES(Manufacturing Execution System).
And when I serialized the data (struct type), something happen.
The code below is what I have done:
[StructLayout(LayoutKind.Sequential, Pack = 4)]
struct DataPackage
{
public int a;
public ushort b;
public byte c;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 5)] public string d;
}
class Program
{
static void Main(string[] args)
{
DataPackage pack1 = new DataPackage();
pack1.a = 0x33333301;
pack1.b = 200;
pack1.c = 21;
pack1.d = "hello";
byte[] pack1_serialized = getBytes(pack1);
Console.WriteLine(BitConverter.ToString(pack1_serialized));
byte[] getBytes(DataPackage str)
{
int size = Marshal.SizeOf(str);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
}
}
And here is the outcome:
I want the outcome to be like this:
33-33-33-01-00-C8-15-68-65-6C-6C-6F
So the questions are:
Why is the uint / ushort type data reverse after Marshalling?
Is there any other way that I can send the data in the sequence that I want ?
Why is the last word "o" in string "hello" disappear in the byte array ?
Thanks.
1 - Because your expected outcome is big endian, and your system appears to use little endian, so basically reversed order of bytes compared to what you expect.
2- Easiest way is to "convert" your numbers to big endian before marshalling (that is change them in a way which will produce desired result while converting them using little endian), for example like this:
static int ToBigEndianInt(int x) {
if (!BitConverter.IsLittleEndian)
return x; // already fine
var ar = BitConverter.GetBytes(x);
Array.Reverse(ar);
return BitConverter.ToInt32(ar, 0);
}
static ushort ToBigEndianShort(ushort x) {
if (!BitConverter.IsLittleEndian)
return x; // already fine
var ar = BitConverter.GetBytes(x);
Array.Reverse(ar);
return BitConverter.ToUInt16(ar, 0);
}
And then:
pack1.a = ToBigEndianInt(0x33333301);
pack1.b = ToBigEndianShort(200);
Note that this way of conversion is not very efficient and if you need more perfomance you can do this with some bit manipulations.
3 - Because string is null terminated, and this null terminator counts in SizeConst. Since you have it 5, there will be 4 characters of your string + 1 null terminator. Just increase SizeConst = 6 (that might add additional zeroes at the end because of Pack = 4).
I'm trying to build a packet structured as:
HEADER OP
Where HEADER size is 4 bytes, and contains total packet length.
OP size is 2 bytes, and contains an integer.
For example, i'm trying to send as OP "3".
So packet should become {2, 0, 0, 0, 3, 0}
Where 2, 0, 0, 0 is header and 3, 0 is OP (padded to the left)
You'll need the following two methods to perform conversions to and from byte arrays. I am assuming this is all Little Endian.
/// <summary>
/// Converts the supplied object to a byte array.
/// </summary>
public static byte[] ToByteArray(object obj)
{
int len = Marshal.SizeOf(obj);
byte[] arr = new byte[len];
IntPtr ptr = Marshal.AllocHGlobal(len);
Marshal.StructureToPtr(obj, ptr, true);
Marshal.Copy(ptr, arr, 0, len);
Marshal.FreeHGlobal(ptr);
return arr;
}
/// <summary>
/// Maps the supplied byte array onto a structure of the specified type.
/// </summary>
public static T ToStructure<T>(byte[] byteArray)
{
GCHandle h = GCHandle.Alloc(byteArray, GCHandleType.Pinned);
T result = (T)Marshal.PtrToStructure(h.AddrOfPinnedObject(), typeof(T));
h.Free();
return result;
}
Now you just need a structure that will hold your data. It will look something like this:
[StructLayout(LayoutKind.Explicit)]
public struct PacketHeader
{
public UInt32 HEADER;
public UInt16 OP;
}
And then all you have to do is populate an instance of PacketHeader with data, and convert it to a byte array like this:
var packetHeader = new PacketHeader
{
HEADER=2,
OP=3
};
var bytes = ToByteArray(packetHeader);
To reverse the process:
var packetHeader p FromByteArray<PacketHeader>(bytes);
Following my other question here, I have been able to share string from C++ to C# thanks to this community.
However, I need to go one level above and I need to share chained structures from C++ to C# using memory mapping.
In an example scenario:
My C++ structures:
struct STRUCT_2
{
char Name[260];
};
struct STRUCT_1
{
void Init()
{
this->Count = 0;
this->Index = 0;
}
DWORD Count;
DWORD Index;
STRUCT_2 Table[256];
};
And me trying to "transfer" it to C#:
[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Ansi)]
public unsafe struct STRUCT_2
{
[FieldOffset(0)]
public fixed char Name[260];
}
[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Ansi)]
public unsafe struct STRUCT_1
{
void Init()
{
this.Count = 0;
this.Index = 0;
}
[FieldOffset(0)]
public uint Count;
[FieldOffset(0)]
public uint Index;
[FieldOffset(100)]
[MarshalAs(UnmanagedType.LPStruct, SizeConst = 256)]
public STRUCT_2 Table;
}
This works partially, basically I can see the values from Count and Index, however I cannot see values or even get them in STRUCT_2.
I have tried to change:
public STRUCT_2[] Table;
But then the compiler tells me:
"The specified Type must be a struct containing no references."
So my question would be, how can you read structures within structures using MemoryMappedFile in C# ?
Advice, thoughts or examples are very welcomed.
Update:
Complete testable code in C#:
[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Ansi)]
public unsafe struct STRUCT_2
{
[FieldOffset(0)]
public fixed byte Name[260];
// Fix thanks to Ben Voigt
}
[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Ansi)]
public unsafe struct STRUCT_1
{
void Init()
{
this.Count = 0;
this.Index = 0;
}
[FieldOffset(0)]
public uint Count;
[FieldOffset(0)]
public uint Index;
[FieldOffset(100)]
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 256)]
public STRUCT_2[] Table;
}
static void Main(string[] args)
{
MemoryMappedFileSecurity CustomSecurity = new MemoryMappedFileSecurity();
CustomSecurity.AddAccessRule(new System.Security.AccessControl.AccessRule<MemoryMappedFileRights>("everyone", MemoryMappedFileRights.FullControl, System.Security.AccessControl.AccessControlType.Allow));
var mappedFile = MemoryMappedFile.CreateOrOpen("Local\\STRUCT_MAPPING", 1024, MemoryMappedFileAccess.ReadWriteExecute, MemoryMappedFileOptions.None, CustomSecurity, System.IO.HandleInheritability.Inheritable);
using (var accessor = mappedFile.CreateViewAccessor())
{
STRUCT_1 data;
accessor.Read<STRUCT_1>(0, out data); // ERROR !
//The specified Type must be a struct containing no references.
Console.WriteLine(data.Count);
Console.WriteLine(data.Index);
}
}
Check this out.
Tested on Visual Studio 2017, Windows 7 x64.
Write and Read data works find.
It's also good study for me.
Take time on this.
Godspeed!
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct STRUCT_2
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 260)]
public byte[] Name;
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 4)]
public struct STRUCT_1
{
void Init()
{
this.Count = 0;
this.Index = 0;
}
public uint Count;
public uint Index;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)] //! array size of 10.
public STRUCT_2 [] Table;
}
static void test3()
{
MemoryMappedFileSecurity CustomSecurity = new MemoryMappedFileSecurity();
CustomSecurity.AddAccessRule(new System.Security.AccessControl.AccessRule<MemoryMappedFileRights>
( "everyone"
, MemoryMappedFileRights.FullControl
, System.Security.AccessControl.AccessControlType.Allow));
using (var mappedFile = MemoryMappedFile.CreateOrOpen("Local\\STRUCT_MAPPING"
, 10 * 1024
, MemoryMappedFileAccess.ReadWriteExecute
, MemoryMappedFileOptions.None
, CustomSecurity
, System.IO.HandleInheritability.Inheritable))
{
using (var accessor = mappedFile.CreateViewAccessor())
{
//! test setting.
int table_count = 5;
//! write data.
STRUCT_1 write_data;
write_data.Index = 1;
write_data.Count = 2;
write_data.Table = new STRUCT_2[10];
for (int i = 0; i < 10; i++)
{
write_data.Table[i].Name = new byte[260];
write_data.Table[i].Name[0] = (byte)i;
}
//! ----------------------------
// Get size of struct
int size = Marshal.SizeOf(typeof(STRUCT_1));
byte[] data = new byte[size];
// Initialize unmanaged memory.
IntPtr p = Marshal.AllocHGlobal(size);
// Copy struct to unmanaged memory.
Marshal.StructureToPtr(write_data, p, false);
// Copy from unmanaged memory to byte array.
Marshal.Copy(p, data, 0, size);
// Write to memory mapped file.
accessor.WriteArray<byte>(0, data, 0, data.Length);
// Free unmanaged memory.
Marshal.FreeHGlobal(p);
p = IntPtr.Zero;
//! ----------------------------------------------
STRUCT_1 read_data;
size = Marshal.SizeOf(typeof(STRUCT_1));
data = new byte[size];
// Initialize unmanaged memory.
p = Marshal.AllocHGlobal(size);
// Read from memory mapped file.
accessor.ReadArray<byte>(0, data, 0, data.Length);
// Copy from byte array to unmanaged memory.
Marshal.Copy(data, 0, p, size);
// Copy unmanaged memory to struct.
read_data = (STRUCT_1)Marshal.PtrToStructure(p, typeof(STRUCT_1));
// Free unmanaged memory.
Marshal.FreeHGlobal(p);
p = IntPtr.Zero;
Console.WriteLine(read_data.Index);
Console.WriteLine(read_data.Count);
}
}
}
Last night I was working on cleaning up some code I found that was a port of an old game I used to play. One of the tricks I used to clean up data was to get rid of the home-brew DataOffsetAttribute that was made for a struct and just make it a plain jane struct and then I can (later) convert it to something a little more useful. Kind of like a person would do with a DataLayer. This worked really good for fixed sized data from a file. Using this method I was about to convert between the two "data types".
public static byte[] ToByteArray<T>(this T dataStructure) where T : struct
{
int size = Marshal.SizeOf(dataStructure);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(dataStructure, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
public static T MarshalAs<T>(this byte[] rawDataStructure) where T : struct
{
var type = typeof(T);
int size = Marshal.SizeOf(type);
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(rawDataStructure, 0, ptr, size);
T structure = (T)Marshal.PtrToStructure(ptr, type);
Marshal.FreeHGlobal(ptr);
return structure;
}
So then I started to wonder if this would work on another project I worked on a long time ago where the data was variable. Here is what I was hoping my data structure would look like
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct RfidReaderResponse
{
public byte PreambleA;
public byte PreambleB;
public byte Length;
public byte LengthLRC;
public ushort ReaderId;
public byte CommandId;
public byte ErrorCode;
public byte[] Data;
public byte LRC;
}
I would probably combine the preamble bytes into a ushort and check if it is a specific value... but that is a different discussion. I have 3 known good responses that I used for testing back in the day. I put those into linqpad as well as my hopeful datastructure. So here is what I am currently using for testing
void Main()
{
var readerResponses = new string[]
{
//data is null because of error EC
"AA-BB-05-05-02-39-0C-EC-21",
//data is 44-00
"AA-BB-07-07-02-39-0C-00-44-00-8B",
//data is 44-00-20-04-13-5E-1A-A4-33-80
"AA-BB-0F-0F-02-39-10-00-44-00-20-04-13-5E-1A-A4-33-80-FB",
};
readerResponses
.Select(x=> x.ToByteArray().MarshalAs<RfidReaderResponse>())
.Dump();
}
now if I comment out the last two fields of the struct I get back what I expect for the first part of the response, but I am just lost on how to get back the data portion. I would prefer to have it as I have it with some magical attribute that the Marshaler understands but so far I can't figure it out. I have tried
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct RfidReaderResponse
{
public byte PreambleA;
public byte PreambleB;
public byte Length;
public byte LengthLRC;
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 2)]
public IntPtr Data;
}
and
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct RfidReaderResponse
{
public byte PreambleA;
public byte PreambleB;
public byte Length;
public byte LengthLRC;
public ushort ReaderId;
public byte CommandId;
public byte ErrorCode;
[MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VarEnum.VT_I1)]
public byte[] Data;
public byte LRC;
}
after a few hours of research. Both of which didn't work stating that my method couldn't make a size for my structure. So I dunno. What am I doing wrong?
EDIT
forgot the method for converting hex strings to byte arrays
public static byte[] ToByteArray(this string hexString)
{
hexString = System.Text.RegularExpressions.Regex.Replace(hexString, "[^0-9A-F.]", "").ToUpper();
if (hexString.Length % 2 == 1)
throw new Exception("The binary key cannot have an odd number of digits");
byte[] arr = new byte[hexString.Length >> 1];
for (int i = 0; i < hexString.Length >> 1; ++i)
{
arr[i] = (byte)((GetHexVal(hexString[i << 1]) << 4) + (GetHexVal(hexString[(i << 1) + 1])));
}
return arr;
}
private static int GetHexVal(char hex)
{
int val = (int)hex;
return val - (val < 58 ? 48 : 55);
}
I tried writing structures to a binary file but I can't read them correctly.
This is my struct. It has a dynamic number of "values". If the number of values is 3, then GetSize() will return 8 + (8*3) = 32
[StructLayout (LayoutKind.Sequential)]
public struct Sample
{
public long timestamp;
public double[] values;
public int GetSize()
{
return sizeof(long) + sizeof(double) * values.Length;
}
}
First, I convert the structure to bytes by:
public static byte[] SampleToBytes(Sample samp)
{
int size = samp.GetSize();
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(samp, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
Then, I write the bytes using BinaryWriter, and exit.
When I have to run the program again and read the file I saved. I use BinaryReader. I get every 32 bytes from the file and convert each array of 32 bytes back to struct using:
public static Sample BytesToSample(byte[] arr)
{
Sample samp = new Sample();
IntPtr ptr = Marshal.AllocHGlobal(32);
Marshal.Copy(arr, 0, ptr, 32);
samp = (Sample)Marshal.PtrToStructure(ptr, samp.GetType());
Marshal.FreeHGlobal(ptr);
return samp;
}
However, a SafeArrayTypeMismatchException occurs at PtrToStructure().
Could anyone please tell me what I am doing wrong?
Thanks.
You do realize that double[], being an array type, is a reference type? The struct holds the long followed by a reference to the heap object.
That would be the reason why it doesn't work, I think.
I believe you should simply write the elements of your array to a binary writer.