I am receiving data packets over UDP that I parse into structs and in a specific situation my data gets all messed up and I can't figure out what the problem might be. Let me give you an overview of what I am doing.
UDP Receiver receives the data packet and passes it to a callback - size of the byte array at this stage is 1307 bytes which is the expected size for the packet
The byte array is marshalled (is that the right term?) into a struct - size of the struct is now 1484 bytes (not sure if that is relevant)
In this example the data is send from the Formula 1 2020 game and the curious thing is that the first entry in the m_carTelemetryData array (see below) is always fine, and the data is all sound. However, every entry of the 22 after the first one is totally messed up with either 0 values, null values or completely outlandish values for all the different fields in the struct.
I tried several things already to pinpoint the issue, but I have now reached the end of my knowledge about the things I am dealing with here. My best guess is that something is going wrong when converting the data into a struct or something else is going on that causes a misalignment (?) of the data.
What I tried so far
Changed my code from "magic marshalling" to manual reading the data byte-by-byte using BinaryReader - no luck
Manually checked the data using BitConverter.ToFoo(bytes, offset) - no luck
Yolo changed the Pack attribute assuming that's where I got wrong - no luck
Double-checked the documentation to make sure I got the data types right - I am fairly confident that I "translated" them correctly
Banged my head against a wall - still no luck
My questions:
Is there something obvious wrong with my code that I am simply not seeing?
Side question: am I wrong in my assumption that the size of the struct should match the size of the byte array it has been created from?
Here is the code for reference (if anything helpful is missing, please let me know):
Packet Header
[StructLayout(LayoutKind.Sequential), Pack = 1]
struct PacketHeader2020 {
public ushort m_packetFormat;
public byte m_gameMajorVersion;
public byte m_gameMinorVersion;
public byte m_packetVersion;
public byte m_packetId;
public ulong m_sessionUUID;
public float m_sessionTime;
public uint m_frameIdentifier;
public byte m_playerCarIndex;
public byte m_secondaryPlayerCarIndex;
}
Packet
public struct CarTelemetryPacket2020
{
public PacketHeader2020 m_header;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 22)]
public CarTelemetryData2020[] m_carTelemetryData;
public ButtonFlag m_buttonStatus;
public byte m_mfdPanelIndex;
public byte m_mfdPanelIndexSecondaryPlayer;
public byte m_suggestedGear;
};
CarTelemetryData2020
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct CarTelemetryData2020
{
public ushort m_speed;
public float m_throttle;
public float m_steer;
public float m_brake;
public byte m_clutch;
public sbyte m_gear;
public ushort m_engineRPM;
public byte m_drs;
public byte m_revLightsPercent;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public ushort[] m_brakesTemperature;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public ushort[] m_tyresSurfaceTemperature;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public ushort[] m_tyresInnerTemperature;
public ushort m_engineTemperature;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public float[] m_tyresPressure;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public SurfaceType[] m_surfaceType;
}
byte[] -> struct
public static T ByteArrayToStructure<T>(byte[] bytes) where T : struct
{
var handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
try
{
return (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
}
finally
{
handle.Free();
}
}
I just pasted your code into LINQPad and found a few problems. The CarTelemetryPacket2020 structure needed a pack clause. Also m_tyresSurfaceTemperature and m_tyresInnerTemperature should have been a byte. The following returns a size of 1307 as per the protocol specification. You're right that the sizes should have matched up and I don't see any other obvious problems with your code.
void Main()
{
System.Runtime.InteropServices.Marshal.SizeOf(typeof(CarTelemetryPacket2020)).Dump();
}
[StructLayout(LayoutKind.Sequential, Pack = 1) ]
public struct PacketHeader2020
{
public ushort m_packetFormat;
public byte m_gameMajorVersion;
public byte m_gameMinorVersion;
public byte m_packetVersion;
public byte m_packetId;
public ulong m_sessionUUID;
public float m_sessionTime;
public uint m_frameIdentifier;
public byte m_playerCarIndex;
public byte m_secondaryPlayerCarIndex;
}
// Added pack = 1
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct CarTelemetryPacket2020
{
public PacketHeader2020 m_header;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 22)]
public CarTelemetryData2020[] m_carTelemetryData;
public UInt32 m_buttonStatus;
public byte m_mfdPanelIndex;
public byte m_mfdPanelIndexSecondaryPlayer;
public byte m_suggestedGear;
};
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct CarTelemetryData2020
{
public ushort m_speed;
public float m_throttle;
public float m_steer;
public float m_brake;
public byte m_clutch;
public sbyte m_gear;
public ushort m_engineRPM;
public byte m_drs;
public byte m_revLightsPercent;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public ushort[] m_brakesTemperature;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
// Changed following to byte
public byte[] m_tyresSurfaceTemperature;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
// Changed following to byte
public byte[] m_tyresInnerTemperature;
public ushort m_engineTemperature;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public float[] m_tyresPressure;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public byte[] m_surfaceType;
}
Related
I'm sure there is a easy answer, but I'm stuck. Been trying to understand deserializing unmanaged data structures into managed data and for simple structs everything is fine, but running into issues when the structs are more complex (Multiple member structs, arrays of structs, etc)
I tried Sequential, Auto and Explicit. Auto and Explicit throw errors, and sequential works but only on simpler types.
Below is my sample code. If I deserialize the binary data as PacketHeader, the info is correct. But If I deserialize as BaseDataPacket, it is not correct. Not even the PacketHeader even though it was when I did it by itself.
I would like to avoid doing explicit because there are potentially a lot of different structs I'd have to put offsets for, but I will if that is the only way.
Main Struct
[Serializable, StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct BaseDataPacket
{
public PacketHeader Header;
public PacketData[] Pdata;
}
Packet Header Struct
[Serializable, StructLayout(LayoutKind.Sequential,Pack =1)]
public struct PacketHeader
{
public ushort Format;
public byte MajorVersion;
public byte MinorVersion;
public byte PacketVersion;
public PacketType PacketId;
public ulong SessionUID;
public float SessionTime;
public uint FrameIdentifier;
public sbyte Index;
public sbyte SecondaryIndex;
}
packet data struct
[Serializable, StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct PacketData
{
public byte d1;
public byte d2;
public byte d3;
public float f1;
public float f2;
public float f3;
public byte b1;
public float f4;
public float f5;
}
And heres my code to marshal
public static T FromBinaryReader<T>(BinaryReader reader)
{
// Read in a byte array
byte[] bytes = reader.ReadBytes(Marshal.SizeOf(typeof(T)));
// Pin the managed memory while, copy it out the data, then unpin it
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
T returnStruct = (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
handle.Free();
return returnStruct;
}
Any thoughts? Maybe this is a duplicate question i didnt find?
Could anyone please help me with the following?
I have a dll written in C and want to call a certain function in it from C#.
The function returns a pointer to a structure. Here is the structure:
typedef struct
{
char crv_name[40];
char crv_name2[12];
char units[40];
char creator[24];
char index_units[8];
double first_dep_tim;
double last_dep_tim;
double level_spacing;
EmptyValU empty_val;
long num_ele;
}
Now, I know (from calling this from a C client) that the last member (num_ele) will be set to 1.
Now, that penultimate member is of type EmptyValU which is defined as:
typedef union
{
double d;
float f;
long l;
ulong ul;
short s;
ushort us;
char c;
}EmptyValU;
Now, I can call this ok fom C# and read everything up until empty_val. My value for num_ele is nonsense as I am clearly misaligned in memory after empty_val.
Here is my C# code:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct CurveInfo
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 40)]
public string crv_name;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 12)]
public string crv_name2;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 40)]
public string units;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 24)]
public string creator;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 8)]
public string index_units;
public double first_dep_tim;
public double last_dep_tim;
public double level_spacing;
[MarshalAs(UnmanagedType.Struct, SizeConst = 8)]
public EmptyValU empty_val;
public long num_ele;
}
and I have defined EmptyValU as:
[StructLayout(LayoutKind.Explicit, Size = 8)]
public struct EmptyValU
{
[FieldOffset(0)]
public double d;
[FieldOffset(0)]
public float f;
[FieldOffset(0)]
long l;
[FieldOffset(0)]
ulong ul;
[FieldOffset(0)]
short s;
[FieldOffset(0)]
ushort us;
[FieldOffset(0)]
char c;
}
Like I say, when I call the function which returns a pointer to such a structure all the values are populated correctly up until the EmptyValU member, which, along with num_ele is not correctly populated. Something about the way I have to define the union in C# is what I am missing in order to keep the alignment correct.
Thanks for any help,
Mitch.
I've solved this.
I C, a long (and ulong) is 4 bytes wide, but in C# they are 8 bytes wide.
So instead of:
[FieldOffset(0)]
long l;
[FieldOffset(0)]
ulong ul;
I should have had:
[FieldOffset(0)]
int l;
[FieldOffset(0)]
uint ul;
becasue in C# int and uint are 4 bytes wide.
For the same reason, on the C# side instead of:
public long num_ele;
I need:
public int num_ele;
Now it all works.
For brevity and clarity the struct and variables have been shortened and renamed.
A BinaryReader is used to populate the following struct:
[StructLayout(LayoutKind.Sequential, Size = 59, CharSet = CharSet.Ansi, Pack = 1)]
public struct TheStruct
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 4)]
public string MyFirstString;
public int MyInt;
public short MyShort;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 8)]
public string MyNextString;
public byte MyByte;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
public string MyLastString;
public double MyDouble;
}
After reading in the data, all variables contain the correct information except the first member MyFirstString.
MyFirstString contains only the first three characters of the four characters that were read.
What is causing this and how do correct it?
Thanks to all who have posted struct examples in the past, they have been a great help to get me this far, but I can't seem to find anything that addresses this issue exactly.
The string is zero terminated, so only SizeConst - 1 characters will be placed in the struct, followed by the terminator character.
You can use a character array to get all characters:
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
public char[] MyFirstString;
I've got this struct :
[StructLayout(LayoutKind.Sequential)]
public struct IS
{
public UInt32 ID;
public UInt32 Quality;
public UInt32 Flags;
public UInt32 Flags2;
public UInt32 ContainerSlots;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public Int32[] ItemStatType;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public UInt32[] ItemStatValue;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public Int32[] ItemStatUnk1;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public Int32[] ItemStatUnk2;
public UInt32 ScalingStatDistribution;
public UInt32 DamageType;
public UInt32 Delay;
public float RangedModRange;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 5)]
public Int32[] SpellId;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 5)]
public Int32[] SpellTrigger;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 5)]
public Int32[] SpellCharges;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 5)]
public Int32[] SpellCooldown;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 5)]
public Int32[] SpellCategory;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 5)]
public Int32[] SpellCategoryCooldown;
public UInt32 Bonding;
public string Name;
public string Name2;
public string Name3;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
public UInt32[] Color;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
public UInt32[] Content;
};
And I'm trying to read bytes from a file and copy those bytes to the above struct using Marshal and a GCHandle, my code is as follows:
reader = BinaryReader.FromFile(fileName);
m_rows = new List<IS>();
int size = Marshal.SizeOf(typeof(IS));
if(reader.BaseStream.Length < size)
return;
byte[] buffer = new byte[size];
buffer = reader.ReadBytes(size);
GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
m_rows.Add((IS)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(IS)));
handle.Free();
But I'm getting an AccessViolationException : attempt to read or write protected memory
I have no idea why this exception is thrown.
I didn't see the bug immediately and wrote a little test program to repro the problem. Using binary search to find the issue, commenting half the fields out repeatedly until I narrowed it down to:
[StructLayout(LayoutKind.Sequential)]
public struct IS {
public string Name;
}
That cannot work, the pinvoke marshaller assumes the default marshaling for string is from a C string, char*. That cannot possibly correct for data that you read from a file, it can never contain valid pointers. The AccessViolation is triggered when it tries to dereference the pointer.
There are no hints in the question to guess how the string was actually serialized to the file. The normal way is:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct IS {
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 42)]
public string Name;
};
Use a hex viewer if necessary to figure out the proper value of SizeConst. If the encoding is unusual (not the system default page) then you have to declare it as byte[] and use the proper Encoding to convert it.
Access violation as you would understand is due to an attempt to read unallocated memory or the memory being released, you may want to check following stack overflow posting:
AccessViolationException when Marshal.PtrToStructure fires
Which refers to the difference between size of managed and native structure as the reason for the issue, essentially you need to provide an offset during marshalling to match the difference between managed and native allocation for the structure.
Check this posting too, where an offset has been added by the user
Access violation exception when use method Marshal.PtrToStructure in a loop.
Another link with a solution:
http://www.codeproject.com/Questions/585390/AccessplusViolationplusException
In case this doesn't help then it is easy to debug such issues using windbg, I can list out the details, in case you need to do it. Also enable Win32 Aces violation exception in VS, it would break at the line throwing exception with some additional information
I'm trying to make this work, but I get this error..
because it contains an object field at offset 0 that is incorrectly aligned or overlapped by a non-object field.
[StructLayout(LayoutKind.Explicit)]
public struct ListEntry {
[System.Runtime.InteropServices.FieldOffset(0)]
[MarshalAs(UnmanagedType.ByValArray, SizeConst=17)]
public byte[] raw;
[System.Runtime.InteropServices.FieldOffset(0)]
public byte version;
[System.Runtime.InteropServices.FieldOffset(1)]
public UInt16 magic;
[System.Runtime.InteropServices.FieldOffset(3)]
public UInt32 start_time;
[System.Runtime.InteropServices.FieldOffset(7)]
public UInt16 run_id;
[System.Runtime.InteropServices.FieldOffset(9)]
public UInt16 channels;
[System.Runtime.InteropServices.FieldOffset(11)]
public UInt16 sampling_rate;
[System.Runtime.InteropServices.FieldOffset(13)]
public UInt32 start_sector;
}
Perhaps as a fixed size buffer?
[System.Runtime.InteropServices.FieldOffset(0)]
public fixed byte raw[17];
Note you'll need to treat that as a byte* in code, for example:
byte* ptr = x.raw;
// now copy / inspect / whatever from ptr