I'm trying to build a packet structured as:
HEADER OP
Where HEADER size is 4 bytes, and contains total packet length.
OP size is 2 bytes, and contains an integer.
For example, i'm trying to send as OP "3".
So packet should become {2, 0, 0, 0, 3, 0}
Where 2, 0, 0, 0 is header and 3, 0 is OP (padded to the left)
You'll need the following two methods to perform conversions to and from byte arrays. I am assuming this is all Little Endian.
/// <summary>
/// Converts the supplied object to a byte array.
/// </summary>
public static byte[] ToByteArray(object obj)
{
int len = Marshal.SizeOf(obj);
byte[] arr = new byte[len];
IntPtr ptr = Marshal.AllocHGlobal(len);
Marshal.StructureToPtr(obj, ptr, true);
Marshal.Copy(ptr, arr, 0, len);
Marshal.FreeHGlobal(ptr);
return arr;
}
/// <summary>
/// Maps the supplied byte array onto a structure of the specified type.
/// </summary>
public static T ToStructure<T>(byte[] byteArray)
{
GCHandle h = GCHandle.Alloc(byteArray, GCHandleType.Pinned);
T result = (T)Marshal.PtrToStructure(h.AddrOfPinnedObject(), typeof(T));
h.Free();
return result;
}
Now you just need a structure that will hold your data. It will look something like this:
[StructLayout(LayoutKind.Explicit)]
public struct PacketHeader
{
public UInt32 HEADER;
public UInt16 OP;
}
And then all you have to do is populate an instance of PacketHeader with data, and convert it to a byte array like this:
var packetHeader = new PacketHeader
{
HEADER=2,
OP=3
};
var bytes = ToByteArray(packetHeader);
To reverse the process:
var packetHeader p FromByteArray<PacketHeader>(bytes);
Related
I want to use my laptop to communicate with MES(Manufacturing Execution System).
And when I serialized the data (struct type), something happen.
The code below is what I have done:
[StructLayout(LayoutKind.Sequential, Pack = 4)]
struct DataPackage
{
public int a;
public ushort b;
public byte c;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 5)] public string d;
}
class Program
{
static void Main(string[] args)
{
DataPackage pack1 = new DataPackage();
pack1.a = 0x33333301;
pack1.b = 200;
pack1.c = 21;
pack1.d = "hello";
byte[] pack1_serialized = getBytes(pack1);
Console.WriteLine(BitConverter.ToString(pack1_serialized));
byte[] getBytes(DataPackage str)
{
int size = Marshal.SizeOf(str);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
}
}
And here is the outcome:
I want the outcome to be like this:
33-33-33-01-00-C8-15-68-65-6C-6C-6F
So the questions are:
Why is the uint / ushort type data reverse after Marshalling?
Is there any other way that I can send the data in the sequence that I want ?
Why is the last word "o" in string "hello" disappear in the byte array ?
Thanks.
1 - Because your expected outcome is big endian, and your system appears to use little endian, so basically reversed order of bytes compared to what you expect.
2- Easiest way is to "convert" your numbers to big endian before marshalling (that is change them in a way which will produce desired result while converting them using little endian), for example like this:
static int ToBigEndianInt(int x) {
if (!BitConverter.IsLittleEndian)
return x; // already fine
var ar = BitConverter.GetBytes(x);
Array.Reverse(ar);
return BitConverter.ToInt32(ar, 0);
}
static ushort ToBigEndianShort(ushort x) {
if (!BitConverter.IsLittleEndian)
return x; // already fine
var ar = BitConverter.GetBytes(x);
Array.Reverse(ar);
return BitConverter.ToUInt16(ar, 0);
}
And then:
pack1.a = ToBigEndianInt(0x33333301);
pack1.b = ToBigEndianShort(200);
Note that this way of conversion is not very efficient and if you need more perfomance you can do this with some bit manipulations.
3 - Because string is null terminated, and this null terminator counts in SizeConst. Since you have it 5, there will be 4 characters of your string + 1 null terminator. Just increase SizeConst = 6 (that might add additional zeroes at the end because of Pack = 4).
Last night I was working on cleaning up some code I found that was a port of an old game I used to play. One of the tricks I used to clean up data was to get rid of the home-brew DataOffsetAttribute that was made for a struct and just make it a plain jane struct and then I can (later) convert it to something a little more useful. Kind of like a person would do with a DataLayer. This worked really good for fixed sized data from a file. Using this method I was about to convert between the two "data types".
public static byte[] ToByteArray<T>(this T dataStructure) where T : struct
{
int size = Marshal.SizeOf(dataStructure);
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(dataStructure, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
public static T MarshalAs<T>(this byte[] rawDataStructure) where T : struct
{
var type = typeof(T);
int size = Marshal.SizeOf(type);
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(rawDataStructure, 0, ptr, size);
T structure = (T)Marshal.PtrToStructure(ptr, type);
Marshal.FreeHGlobal(ptr);
return structure;
}
So then I started to wonder if this would work on another project I worked on a long time ago where the data was variable. Here is what I was hoping my data structure would look like
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct RfidReaderResponse
{
public byte PreambleA;
public byte PreambleB;
public byte Length;
public byte LengthLRC;
public ushort ReaderId;
public byte CommandId;
public byte ErrorCode;
public byte[] Data;
public byte LRC;
}
I would probably combine the preamble bytes into a ushort and check if it is a specific value... but that is a different discussion. I have 3 known good responses that I used for testing back in the day. I put those into linqpad as well as my hopeful datastructure. So here is what I am currently using for testing
void Main()
{
var readerResponses = new string[]
{
//data is null because of error EC
"AA-BB-05-05-02-39-0C-EC-21",
//data is 44-00
"AA-BB-07-07-02-39-0C-00-44-00-8B",
//data is 44-00-20-04-13-5E-1A-A4-33-80
"AA-BB-0F-0F-02-39-10-00-44-00-20-04-13-5E-1A-A4-33-80-FB",
};
readerResponses
.Select(x=> x.ToByteArray().MarshalAs<RfidReaderResponse>())
.Dump();
}
now if I comment out the last two fields of the struct I get back what I expect for the first part of the response, but I am just lost on how to get back the data portion. I would prefer to have it as I have it with some magical attribute that the Marshaler understands but so far I can't figure it out. I have tried
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct RfidReaderResponse
{
public byte PreambleA;
public byte PreambleB;
public byte Length;
public byte LengthLRC;
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 2)]
public IntPtr Data;
}
and
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct RfidReaderResponse
{
public byte PreambleA;
public byte PreambleB;
public byte Length;
public byte LengthLRC;
public ushort ReaderId;
public byte CommandId;
public byte ErrorCode;
[MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VarEnum.VT_I1)]
public byte[] Data;
public byte LRC;
}
after a few hours of research. Both of which didn't work stating that my method couldn't make a size for my structure. So I dunno. What am I doing wrong?
EDIT
forgot the method for converting hex strings to byte arrays
public static byte[] ToByteArray(this string hexString)
{
hexString = System.Text.RegularExpressions.Regex.Replace(hexString, "[^0-9A-F.]", "").ToUpper();
if (hexString.Length % 2 == 1)
throw new Exception("The binary key cannot have an odd number of digits");
byte[] arr = new byte[hexString.Length >> 1];
for (int i = 0; i < hexString.Length >> 1; ++i)
{
arr[i] = (byte)((GetHexVal(hexString[i << 1]) << 4) + (GetHexVal(hexString[(i << 1) + 1])));
}
return arr;
}
private static int GetHexVal(char hex)
{
int val = (int)hex;
return val - (val < 58 ? 48 : 55);
}
Code For NET 2.0.
I write function ByteArrayToObject for insert offset bytes in struct, but is it possible to quickly?
Is planned that there will be a lot of structures in which it is necessary to append the changed network information. If I can insert these bytes quickly to the right place, it will be organized in the protocol as one big structure.
Thank you for any help.
In my case, I do not like that every time to replace the bytes that have to do all the copy of the object func ObjectToByteArray.
/// <summary> Convert an object struct to a byte array </summary>
private static byte[] ObjectToByteArray(Object obj)
{
var size = Marshal.SizeOf(obj);
// Both managed and unmanaged buffers required.
var bytes = new byte[size];
var ptr = Marshal.AllocHGlobal(size);
// Copy object byte-to-byte to unmanaged memory.
Marshal.StructureToPtr(obj, ptr, false);
// Copy data from unmanaged memory to managed buffer.
Marshal.Copy(ptr, bytes, 0, size);
// Release unmanaged memory.
Marshal.FreeHGlobal(ptr);
return bytes;
}
/// <summary> Need Faster ? </summary>
public static T ByteArrayToObject<T>(ref T obj, int StartOffset, params byte[] bytes)
{
int size = Marshal.SizeOf(obj);
int Length = (bytes.Length > size) ? size : bytes.Length;
byte[] Allbytes = ObjectToByteArray(obj);
Array.Copy(bytes, 0, Allbytes, StartOffset, Length - StartOffset);
var ptr = Marshal.AllocHGlobal(size);
Marshal.Copy(Allbytes, 0, ptr, Length );
obj = (T)Marshal.PtrToStructure(ptr, typeof(T));
Marshal.FreeHGlobal(ptr);
return obj;
}
Example use
[Serializable]
[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential, Pack = 1, CharSet = System.Runtime.InteropServices.CharSet.Ansi)]
struct Protocol
{
public byte f0;
public byte f1;
public short f2;
public byte f3;
public long f4;
[System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst = 20000)]
public int[] Array; // 20000
}
System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
for (byte i = 1; i < 10; i++)
{
sw.Reset();
sw.Start();
ob = ByteArrayToObject<Protocol>(ref ob,1, i, 0x11, i, 0x22, i);
sw.Stop();
Console.WriteLine("Tick =" + sw.ElapsedTicks);
}
Output
Tick =9940
Tick =686
Tick =593
Tick =474
Tick =562
Tick =5283
Tick =193
Tick =173
Tick =164
This is too long for a comment, but to expand on the unsafe approach:
unsafe struct Ex
{
public byte f0,f1,f2,f3,f4;
public fixed int buffer[20000];
}
class Program
{
public static unsafe void ByteArrayToEx(Ex* obj, int offset, params byte[] bytes)
{
// you should add some safely nets here sizeof(Ex) should used for size of struct
byte* p = (byte*)obj;
foreach (var b in bytes)
{
p[offset++] = b;
}
// dont return value, it is expensive!
}
unsafe static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
Console.WriteLine(Stopwatch.Frequency);
Ex e = new Ex { f0 = 0, f1 = 1, f2 = 2, f3 = 3, f4 = 4 };
ByteArrayToEx(&e, 2, 5, 6, 7);
for (int i = 0; i < 10; i++) {
sw.Restart();
ByteArrayToEx(&e, 2, (byte) i, 6, 7);
sw.Stop();
Console.WriteLine(sw.ElapsedTicks);
}
}
}
This may or may not work for you. Also dont return the value. You are already mutating the pointer to it. Returning a copy of such a big struct adds 10 ticks to every call to it.
Also, you need to do at least 1 warmup when bench marking. That is why the first number is so poor.
Results on my PC:
3312929
4
2
0
0
0
0
0
0
0
0
Rewrote a little
public static unsafe void ByteArrayToEx(ref Ex value, int offset, params byte[] bytes)
{
// you should add some safely nets here sizeof(Ex) should used for size of struct
fixed (Ex* obj = &value)
{
byte* p = (byte*)obj;
foreach (var b in bytes)
{
p[offset++] = b;
}
}
// dont return value, it is expensive!
}
I tried writing structures to a binary file but I can't read them correctly.
This is my struct. It has a dynamic number of "values". If the number of values is 3, then GetSize() will return 8 + (8*3) = 32
[StructLayout (LayoutKind.Sequential)]
public struct Sample
{
public long timestamp;
public double[] values;
public int GetSize()
{
return sizeof(long) + sizeof(double) * values.Length;
}
}
First, I convert the structure to bytes by:
public static byte[] SampleToBytes(Sample samp)
{
int size = samp.GetSize();
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(samp, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);
return arr;
}
Then, I write the bytes using BinaryWriter, and exit.
When I have to run the program again and read the file I saved. I use BinaryReader. I get every 32 bytes from the file and convert each array of 32 bytes back to struct using:
public static Sample BytesToSample(byte[] arr)
{
Sample samp = new Sample();
IntPtr ptr = Marshal.AllocHGlobal(32);
Marshal.Copy(arr, 0, ptr, 32);
samp = (Sample)Marshal.PtrToStructure(ptr, samp.GetType());
Marshal.FreeHGlobal(ptr);
return samp;
}
However, a SafeArrayTypeMismatchException occurs at PtrToStructure().
Could anyone please tell me what I am doing wrong?
Thanks.
You do realize that double[], being an array type, is a reference type? The struct holds the long followed by a reference to the heap object.
That would be the reason why it doesn't work, I think.
I believe you should simply write the elements of your array to a binary writer.
I have a C# app which read/write messages of some predefined protocol from/to USB. So I have at least 2 options here. One is serialize/Deserialize. The other is marshal. At the beginning I was picking the first option and it worked fine. But after more and more message types are introduced I am annoyed by the cumbersome implementation for each message type and feel the marshal may be a better way to go. I haven't use marshal before. Is it the right way to go? I did some test. One problem I have is when I have a structure with an array, how to write to it? Such as the following:-
[StructLayout(LayoutKind.Sequential, Size=TotalBytesInStruct),Serializable]
public struct SomeData
{
/// char[15]
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 15)]
public sbyte[] data;
/// int[15]
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 15)]
public int[] prob;
}
It appears that the data and prob are both reference and I have to new a object to use them, which sounds not quite right to me. I need the whole struct to be a continuous block of memory, which I don't think the new operator will do that for me.
Any suggestion is really appreciated
The following is how I did by serialization.
List<byte> rawData = new List<byte>();
rawData.AddRange(BitConverter.GetBytes(ProtocolVersion));
// 16 bytes for operator ID
byte[] temp = new byte[16];
Array.Copy(Encoding.ASCII.GetBytes(OperatorId), temp, OperatorId.Length > temp.Length ? temp.Length : OperatorId.Length);
rawData.AddRange(temp);
// 16 bytes for operator password
Array.Clear(temp, 0, temp.Length);
Array.Copy(Encoding.ASCII.GetBytes(OperatorPassword), temp, OperatorPassword.Length > temp.Length ? temp.Length : OperatorPassword.Length);
rawData.AddRange(temp);
The following is how I did by marshaling
static byte[] RawSerialize<T>(T obj)
{
if (obj == null)
{
return null;
}
int size = Marshal.SizeOf(obj);
byte[] result = new byte[size];
IntPtr buffer = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(obj, buffer, false);
Marshal.Copy(buffer, result, 0, size);
return result;
}
It seems that I only need to new an array like the following and it works fine.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class SomeData
{
/// char[15]
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 15)]
public byte[] data = new byte[15];
/// int[15]
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 15)]
public int[] prob = new int[15];
}