I have a C struct in an embedded MCU with about 1000 elements, and it contains lots of fixed size arrays and other structs inside, Now I want to bring the data to PC Using C#
Here is a simple preview of my struct elements in C
struct _registers
{
char name[32];
float calibrate[4][16];
float DMTI;
float DMTII;
float DMTIII;
float DMTIE;
float DMTIIE;
....
};
Now I want to convert the Struct into C# using the GCHandle class,
something like this
//The C struct is in this byte array named buffer
byte[] buffer = new byte[4096];
GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
_registers stuff = (protection_registers)Marshal.PtrToStructure(handle.AddrOfPinnedObject(),typeof(_registers));
handle.Free();
the problem is that the Visual studio complains about the "Pointers and fixed size buffers may only be used in an unsafe context"
is there a way to use it normally without unsafe code? I have found Doing something like this
[StructLayout(LayoutKind.Explicit, Size = 56, Pack = 1)]
public struct NewStuff
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
[FieldOffset(0)]
public string name;
[MarshalAs(UnmanagedType.U4)]
[FieldOffset(32)]
public float calibrate[4][16];
}
but as the Code on the MCU is evolving in the coming years and we will add lots of functionality and parameter to the Struct, and since the struct has already 1000 elements, how we can do it better and more clever way? because keep tracking of all the offsets is very hard and error prone!
Try doing something like this instead (note: using class instead of struct which is more appropriate for C# - still marshals OK to a C++ struct):
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public class NewStuff
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
public StringBuilder name;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 4*16)]
public float[,] calibrate;
[MarshalAs(UnmanagedType.R4)]
public float DMTI;
[MarshalAs(UnmanagedType.R4)]
public float DMTII;
// Etc
}
You won't be able to remove the SizeConst since the marshalling needs to know this.
Also, when you intialise the class, you will need to set the array fields to the appropriate sized buffers, and initialise the StringBuilder with the correct buffer size.
Doing it this way means you can avoid using fixed buffers (and hence, you avoid the unsafe code).
Related
I'm debugging code in C# written by a 3rd party. The project is an old C++ project that was rewritten in C# by a contractor, and I have no access to the contractor. I authored the original C++ version.
The issue is when the C# code gets the size of a structure that represents data received over a UDP connection.
The struct is defined as:
[StructLayout(LayoutKind.Sequential,Pack=1)]
internal class PROXY_HDR {
public ushort pad;
public ushort label;
public char flags;
public ushort length;
public char[] ip = new char[4];
public ushort port;
}
The size of this struct is retrieved as:
int size = Marshal.Sizeof(typeof(PROXY_HDR));
and the value returned is 17 instead of the expected 13. With a 4-byte difference, I suspect the ip member, but only because it's expressed differently than the other members (with 'new'), but I have no other basis to decide.
I don't typically use this type of marshaling in my C# code that parses received packets without a problem, so I don't know how to modify this struct definition to make it 'line up' with the original version size-wise.
I could replace the Marshal line with
int size = 13;
but that's cheating, right?
Can I modify this layout somehow to get the size to come out right?
Add this to the structure:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
internal class PROXY_HDR
{
public ushort pad;
public ushort label;
public byte flags;
public ushort length;
[MarshalAs(UnmanagedType.ByValArray,
SizeConst = 4, ArraySubType = UnmanagedType.U1)]
public byte[] ip;
public ushort port;
}
This will tell the compiler to treat it as a typical C-style array, not a pointer. This should be byte, as an IP address is an unsigned char array. char typically isn't used in these types of headers, it's usually byte (i.e: unsigned char)
I've seen the attempted to read or write protected memory error before.
Typically the error shows up when I don't set up the c# struct correctly. I do have other calls working properly but this one is not co-operating.
I'm almost certain that it could be both my function call and the struct that is causing the problem.
C Syntax
int CardTransaction(pTRequest req, char *ProductCodes)
Request structure (I condensed it b/c there were repetitive data types)
typedef struct _cardRequest
{
unsigned short RedemptionNum
long TotalAmount;
unsigned char filler1[257];
char CardNumber[80];
unsigned char cardType;
} TRequest, *pTRequest;
C# function call
[DllImport("card.dll"), CallingConvention.Cdecl, CharSet = CharSet.Auto)]
public static extern int CardTransaction(ref CardRequest cardRequest, [MarshalAs(UnManagedType.LPStr)] StringBuilder productCodes);
ProductCodes is null so I just instantiated a stringbuilder object with nothing in it and passed it through. This is one place I'm thinking could be a problem.
C# structure
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct CardRequest
{
public uint16 RedemptionNum
public int TotalAmount;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 257)]
public string filler1;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 80)]
public string CardNumber;
public byte cardType;
}
The obvious problem is that the C code uses an aligned struct, but for some reason you have elected to pack the C# struct. Remove the Pack = 1 from the C# code to make the two structures match.
Beyond that the filler array looks more like a byte array than a string. I'd declare it like this:
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 257)]
public byte[] filler1;
If you want to pass null to the productCodes parameter, then I expect you can do just that. I cannot recall every doing that myself, but generally when you pass null to a p/invoke, then the marshaller will pass NULL to the native code.
I am trying to call an unmanaged C++ function, that has a structure as an input parameter.
The structure is defined in the header file like this:
struct MyStruct
{
int siOrder;
char aaszNames[6][25];
int siId[6];
int siTones[6];
};
I tried to declare the managed struct as following:
[StructLayoutAttribute(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public struct MyStruct {
public int siOrder;
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst=150)]
public string aaszNames;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst=6, ArraySubType=UnmanagedType.I4)]
public int[] siId;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst=6, ArraySubType=UnmanagedType.I4)]
public int[] siTones;
}
But without any success. I am guessing that the marshaling fails, since the aaszNames is actually an array of six 25 long null-terminating strings.
I tried declaring aaszNames as
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst=150)]
public char[] aaszNames;
filling the array with nulls where necessary. But, again, nothing.
Is there something I am missing? What am I dong wrong? What is the best way to marshal this 2-D char array?
Any hints, please.
Try using multiple C# structs:
[StructLayoutAttribute(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public struct MyStruct_Name
{
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 25)]
public string name;
}
[StructLayoutAttribute(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct MyStruct
{
public int siOrder;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 6)]
public MyStruct_Name aaszNames;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 6, ArraySubType = UnmanagedType.I4)]
public int[] siId;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 6, ArraySubType = UnmanagedType.I4)]
public int[] siTones;
}
This is how I've been passing arrays of C-style strings around.
Don't forget to create the contents of aaszNames! The marshaller hates null references.
MyStruct foo = new MyStruct();
for (int i = 0; i < 6; i++)
{
foo.aaszNames[i] = new MyStruct_Name();
foo.aaszNames[i].name = "";
}
Good luck!
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst=150)]
public char[] aaszNames;
That marshalling type looks well. Probably issue in function call, or bad memory allocation/
I would write a small c-program to check the byte size of the C-structure.
Then I would go with the other suggestion to extract the data field by field.
From a C standpoint the /0 is treated as normal character included in the 6 bytes whereas C# would use a length of 5 and have the /0 hidden.
char aaszNames[6][25];
char of C++ Type is 8 bits~
but char of C# Type is Unicode ,(16 bits) !
so char of C++ Type <-> byte of C# type
I am doing some C# interop work. I have the following struct:
#pragma pack(push,1)
typedef struct
{
unsigned __int64 Handle;
LinkType_t Type;
LinkState_t State;
unsigned __int64 Settings;
signed __int8 Name[MAX_LINK_NAME];
unsigned __int8 DeviceInfo[MAX_LINK_DEVINFO];
unsigned __int8 Reserved[40];
} LinkInfo_t;
This is my attempt to convert it into a C# struct:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct LinkInfo_t
{
[MarshalAs(UnmanagedType.U8)]
public UInt64 Handle;
[MarshalAs(UnmanagedType.I4)]
public LinkType_t Type;
[MarshalAs(UnmanagedType.I4)]
public LinkState_t State;
[MarshalAs(UnmanagedType.U8)]
public UInt64 Settings;
[MarshalAs(UnmanagedType.LPStr, SizeConst = MAX_LINK_NAME)]
public string Name;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = MAX_LINK_DEVINFO, ArraySubType = UnmanagedType.U1)]
public byte[] DeviceInfo;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 40, ArraySubType = UnmanagedType.U1)]
public byte[] Reserved;
}
However, whenever I initialize the struct the Name, DeviceInfo, and Reserved fields are all set to null. How do I fix this?
For the arrays, try to use the fixed keyword:
public fixed byte DeviceInfo[MAX_LINK_DEVINFO];
public fixed byte Reserved[40];
whenever I initialize the struct the
Name, DeviceInfo, and Reserved fields
are all set to null
This is correct, and your definition looks OK to me (BTW, you don't need [MarshalAs] on the primitive fields, the default behaviour is to do what you specified there). Because your array fields are null, the marshaler won't do anything about them when marshaling your struct to unmanaged memory, but it's going to create the strings and arrays when unmarshaling.
What Anton Tykhyy says is correct. I just want to clarify with some examples. Using 'fixed' works, but that forces you to use 'unsafe' as well. I like to avoid using unsafe wherever possible. Using Marshal is a way to get around that.
First, let's say that I have a library that was created in C with the following definitions.
typedef struct {
int messageType;
BYTE payload[60];
} my_message;
/**
* \param[out] msg Where the message will be written to
*/
void receiveMessage(my_message *msg);
/*
* \param[in] msg The message that will be sent
*/
void sendMessage(my_message *msg);
In C#, the following structure would be equivalent to the one in C.
[StructLayout(LayoutKind.Sequential, Size = 64), Serializable]
struct my_message
{
int messageType;
[MarshalAs(UnmanagedType.ByValArray,SizeConst = 60)]
byte[] payload;
public initializeArray()
{
//explicitly initialize the array
payload = new byte[60];
}
}
Since the msg in receiveMessage() is documented as [out], you don't need to do anything special to the array in the structure before passing it to the function. i.e.:
my_message msg = new my_message();
receiveMessage(ref msg);
byte payload10 = msg.payload[10];
Since the msg in sendMessage() is documented as [in], you will need to fill the array before calling the function. Before filling the array, the array needs to be explicitly instantiated before using it. i.e.:
my_message msg = new my_message();
msg.initializeArray();
msg.payload[10] = 255;
sendMessage(ref msg);
Calling initializeArray() should instantiate the array in the previously allocated space created within the struct for this array.
I'm trying to read binary data using C#. I have all the information about the layout of the data in the files I want to read. I'm able to read the data "chunk by chunk", i.e. getting the first 40 bytes of data converting it to a string, get the next 40 bytes.
Since there are at least three slightly different version of the data, I would like to read the data directly into a struct. It just feels so much more right than by reading it "line by line".
I have tried the following approach but to no avail:
StructType aStruct;
int count = Marshal.SizeOf(typeof(StructType));
byte[] readBuffer = new byte[count];
BinaryReader reader = new BinaryReader(stream);
readBuffer = reader.ReadBytes(count);
GCHandle handle = GCHandle.Alloc(readBuffer, GCHandleType.Pinned);
aStruct = (StructType) Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(StructType));
handle.Free();
The stream is an opened FileStream from which I have began to read from. I get an AccessViolationException when using Marshal.PtrToStructure.
The stream contains more information than I'm trying to read since I'm not interested in data at the end of the file.
The struct is defined like:
[StructLayout(LayoutKind.Explicit)]
struct StructType
{
[FieldOffset(0)]
public string FileDate;
[FieldOffset(8)]
public string FileTime;
[FieldOffset(16)]
public int Id1;
[FieldOffset(20)]
public string Id2;
}
The examples code is changed from original to make this question shorter.
How would I read binary data from a file into a struct?
The problem is the strings in your struct. I found that marshaling types like byte/short/int is not a problem; but when you need to marshal into a complex type such as a string, you need your struct to explicitly mimic an unmanaged type. You can do this with the MarshalAs attrib.
For your example, the following should work:
[StructLayout(LayoutKind.Explicit)]
struct StructType
{
[FieldOffset(0)]
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 8)]
public string FileDate;
[FieldOffset(8)]
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 8)]
public string FileTime;
[FieldOffset(16)]
public int Id1;
[FieldOffset(20)]
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 66)] //Or however long Id2 is.
public string Id2;
}
Here is what I am using.This worked successfully for me for reading Portable Executable Format.It's a generic function, so T is your struct type.
public static T ByteToType<T>(BinaryReader reader)
{
byte[] bytes = reader.ReadBytes(Marshal.SizeOf(typeof(T)));
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
T theStructure = (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
handle.Free();
return theStructure;
}
As Ronnie said, I'd use BinaryReader and read each field individually. I can't find the link to the article with this info, but it's been observed that using BinaryReader to read each individual field can be faster than Marshal.PtrToStruct, if the struct contains less than 30-40 or so fields. I'll post the link to the article when I find it.
The article's link is at: http://www.codeproject.com/Articles/10750/Fast-Binary-File-Reading-with-C
When marshaling an array of structs, PtrToStruct gains the upper-hand more quickly, because you can think of the field count as fields * array length.
I don't see any problem with your code.
just out of my head, what if you try to do it manually? does it work?
BinaryReader reader = new BinaryReader(stream);
StructType o = new StructType();
o.FileDate = Encoding.ASCII.GetString(reader.ReadBytes(8));
o.FileTime = Encoding.ASCII.GetString(reader.ReadBytes(8));
...
...
...
also try
StructType o = new StructType();
byte[] buffer = new byte[Marshal.SizeOf(typeof(StructType))];
GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
Marshal.StructureToPtr(o, handle.AddrOfPinnedObject(), false);
handle.Free();
then use buffer[] in your BinaryReader instead of reading data from FileStream to see whether you still get AccessViolation exception.
I had no luck using the
BinaryFormatter, I guess I have to
have a complete struct that matches
the content of the file exactly.
That makes sense, BinaryFormatter has its own data format, completely incompatible with yours.
I had no luck using the BinaryFormatter, I guess I have to have a complete struct that matches the content of the file exactly. I realised that in the end I wasn't interested in very much of the file content anyway so I went with the solution of reading part of stream into a bytebuffer and then converting it using
Encoding.ASCII.GetString()
for strings and
BitConverter.ToInt32()
for the integers.
I will need to be able to parse more of the file later on but for this version I got away with just a couple of lines of code.
Try this:
using (FileStream stream = new FileStream(fileName, FileMode.Open))
{
BinaryFormatter formatter = new BinaryFormatter();
StructType aStruct = (StructType)formatter.Deserialize(filestream);
}
Reading straight into structs is evil - many a C program has fallen over because of different byte orderings, different compiler implementations of fields, packing, word size.......
You are best of serialising and deserialising byte by byte. Use the build in stuff if you want or just get used to BinaryReader.
I had structure:
[StructLayout(LayoutKind.Explicit, Size = 21)]
public struct RecordStruct
{
[FieldOffset(0)]
public double Var1;
[FieldOffset(8)]
public byte var2
[FieldOffset(9)]
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 12)]
public string String1;
}
}
and I received "incorrectly aligned or overlapped by non-object".
Based on that I found:
https://social.msdn.microsoft.com/Forums/vstudio/en-US/2f9ffce5-4c64-4ea7-a994-06b372b28c39/strange-issue-with-layoutkindexplicit?forum=clr
OK. I think I understand what's going on here. It seems like the
problem is related to the fact that the array type (which is an object
type) must be stored at a 4-byte boundary in memory. However, what
you're really trying to do is serialize the 6 bytes separately.
I think the problem is the mix between FieldOffset and serialization
rules. I'm thinking that structlayout.sequential may work for you,
since it doesn't actually modify the in-memory representation of the
structure. I think FieldOffset is actually modifying the in-memory
layout of the type. This causes problems because the .NET framework
requires object references to be aligned on appropriate boundaries (it
seems).
So my struct was defined as explicit with:
[StructLayout(LayoutKind.Explicit, Size = 21)]
and thus my fields had specified
[FieldOffset(<offset_number>)]
but when you change your struct to Sequentional, you can get rid of those offsets and the error will disappear. Something like:
[StructLayout(LayoutKind.Sequential, Size = 21)]
public struct RecordStruct
{
public double Var1;
public byte var2;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 12)]
public string String1;
}
}