Develop Reference

Marshalling an empty void* parameter that can take multiple structure and retrieving it in C#

I want to call a C function with an empty void* parameter and retrieve it in C# to print the struct values.
In this example:
The C code will randomly assign one of the two structures to the void parameter.
The C code return must be an int.
The C code structures can be modified
How to retrieve and print the C struct content in C#?
C
#include <time.h>
#include <stdlib.h>
void PrintKdesc(interTICKeyDescription* kdescr)
{
kdescr->nItems++;
printf("nItems = %d\n", kdescr->nItems);
}
void PrintKevent(interTICEvent* kevent)
{
kevent->nbr++;
printf("nbr = %d\n", kevent->nbr);
}
EXPORT int VoidPtrTest(void* test)
{
interTICKeyDescription kdesc = { 1, 0x250021, 0, 42 };
interTICEvent kevent = { 2, 8 };
srand(time(NULL));
int r = rand() % 2;
if (r == 1)
{
test = &kdesc;
PrintKdesc(test);
}
else
{
test = &kevent;
PrintKevent(test);
}
return (0);
}
typedef struct {
int id;
unsigned int networkId;
unsigned int implVersion;
int nItems;
} interTICKeyDescription;
typedef struct {
int id;
int nbr;
} interTICEvent;
C#
[StructLayout(LayoutKind.Sequential)]
public struct InterTICKeyDescription
{
public int id;
public UInt32 networkId;
public UInt32 implVersion;
public int nItems;
}
[StructLayout(LayoutKind.Sequential)]
public struct InterTICEvent
{
public int id;
public int nbr;
}
[DllImport("Sandbox.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
extern static int VoidPtrTest(IntPtr test);
My attempt
static void Main()
{
InterTICKeyDescription kdesc = new InterTICKeyDescription();
InterTICEvent kevent = new InterTICEvent();
IntPtr ptr = Marshal.AllocHGlobal(Marshal.SizeOf(kdesc));
VoidPtrTest(ptr);
int id = Marshal.ReadInt32(ptr, 0);
if (id == 1)
{
kdesc = (InterTICKeyDescription)Marshal.PtrToStructure(ptr, typeof(InterTICKeyDescription));
Console.WriteLine("Id 1: " + kdesc.nItems);
}
else
{
kevent = (InterTICEvent)Marshal.PtrToStructure(ptr, typeof(InterTICEvent));
Console.WriteLine("Id 2: " + kevent.nbr);
}
Marshal.FreeHGlobal(ptr);
}

Detect the presence of multiple CPU kGroups or NUMA nodes

I would like to write a tool in C# which can detect the presence of multiple CPU groups in Windows. I have seen this referred to as kGroups, and also as NUMA nodes.
This need has grown out of multiple performance related issues where we discovered the customer was running HP servers which often have their NUMA Group Size Optimization in the BIOS set to "Clustered" instead of "Flat" which can result in multiple CPU groups in Windows. Any one process, unless otherwise designed to operate across kGroups, will only be able to use logical processors within the kGroup the process affinity is set to.
I have found a number of resources that can detect information about number of physical/logical processors, but I'm unable to find information on if/how those CPU's might be logically grouped. I'm open to getting this info through p/invoke or WMI among other methods.
Edit: Found the following post with a full example of the GetLogicalProcessorInformationEx call via p/invoke. Will update when I can confirm how to test numa node configuration.
https://stackoverflow.com/a/6972620/3736007
references: http://h17007.www1.hpe.com/docs/iss/proliant_uefi/UEFI_Gen9_060216/s_set_NUMA_group.html
Solved (I think). I was able to use David Heffernan's c# implementation, wrapped it up in a class and added some properties to return some of the information I'm after. Unsafe code is still a bit of black magic to me so I know it could be done better, but it's working so far.
public static class LogicalProcessorInformation
{
public static int CpuPackages
{
get
{
if (_buffer == null)
_buffer = MyGetLogicalProcessorInformation();
return _buffer.Count(b => b.Relationship == LOGICAL_PROCESSOR_RELATIONSHIP.RelationProcessorPackage);
}
}
public static int CpuCores
{
get
{
if (_buffer == null)
_buffer = MyGetLogicalProcessorInformation();
return _buffer.Count(b => b.Relationship == LOGICAL_PROCESSOR_RELATIONSHIP.RelationProcessorCore);
}
}
public static int LogicalProcessors
{
get
{
if (_buffer == null)
_buffer = MyGetLogicalProcessorInformation();
var count = 0;
foreach (var obj in _buffer.Where(b => b.Relationship == LOGICAL_PROCESSOR_RELATIONSHIP.RelationProcessorCore))
{
count += CountSetBits(obj.ProcessorMask);
}
return count;
}
}
public static int CpuGroups
{
get
{
if (_buffer == null)
_buffer = MyGetLogicalProcessorInformation();
return _buffer.Count(b => b.Relationship == LOGICAL_PROCESSOR_RELATIONSHIP.RelationGroup);
}
}
public static int NumaNodes
{
get
{
if (_buffer == null)
_buffer = MyGetLogicalProcessorInformation();
return _buffer.Count(b => b.Relationship == LOGICAL_PROCESSOR_RELATIONSHIP.RelationNumaNode);
}
}
private static int CountSetBits(UIntPtr bitMask)
{
// todo: get rid of tostring and figure out the right way.
var bitMaskuint = uint.Parse(bitMask.ToString());
uint count = 0;
while (bitMaskuint != 0)
{
count += bitMaskuint & 1;
bitMaskuint >>= 1;
}
return (int)count;
}
private static SYSTEM_LOGICAL_PROCESSOR_INFORMATION[] _buffer;
[StructLayout(LayoutKind.Sequential)]
private struct PROCESSORCORE
{
public byte Flags;
};
[StructLayout(LayoutKind.Sequential)]
private struct NUMANODE
{
public uint NodeNumber;
}
private enum PROCESSOR_CACHE_TYPE
{
CacheUnified,
CacheInstruction,
CacheData,
CacheTrace
}
[StructLayout(LayoutKind.Sequential)]
private struct CACHE_DESCRIPTOR
{
public byte Level;
public byte Associativity;
public ushort LineSize;
public uint Size;
public PROCESSOR_CACHE_TYPE Type;
}
[StructLayout(LayoutKind.Explicit)]
private struct SYSTEM_LOGICAL_PROCESSOR_INFORMATION_UNION
{
[FieldOffset(0)]
public PROCESSORCORE ProcessorCore;
[FieldOffset(0)]
public NUMANODE NumaNode;
[FieldOffset(0)]
public CACHE_DESCRIPTOR Cache;
[FieldOffset(0)]
private UInt64 Reserved1;
[FieldOffset(8)]
private UInt64 Reserved2;
}
private enum LOGICAL_PROCESSOR_RELATIONSHIP
{
RelationProcessorCore,
RelationNumaNode,
RelationCache,
RelationProcessorPackage,
RelationGroup,
RelationAll = 0xffff
}
private struct SYSTEM_LOGICAL_PROCESSOR_INFORMATION
{
public UIntPtr ProcessorMask;
public LOGICAL_PROCESSOR_RELATIONSHIP Relationship;
public SYSTEM_LOGICAL_PROCESSOR_INFORMATION_UNION ProcessorInformation;
}
[DllImport(#"kernel32.dll", SetLastError = true)]
private static extern bool GetLogicalProcessorInformation(
IntPtr Buffer,
ref uint ReturnLength
);
private const int ERROR_INSUFFICIENT_BUFFER = 122;
private static SYSTEM_LOGICAL_PROCESSOR_INFORMATION[] MyGetLogicalProcessorInformation()
{
uint ReturnLength = 0;
GetLogicalProcessorInformation(IntPtr.Zero, ref ReturnLength);
if (Marshal.GetLastWin32Error() == ERROR_INSUFFICIENT_BUFFER)
{
IntPtr Ptr = Marshal.AllocHGlobal((int)ReturnLength);
try
{
if (GetLogicalProcessorInformation(Ptr, ref ReturnLength))
{
int size = Marshal.SizeOf(typeof(SYSTEM_LOGICAL_PROCESSOR_INFORMATION));
int len = (int)ReturnLength / size;
SYSTEM_LOGICAL_PROCESSOR_INFORMATION[] Buffer = new SYSTEM_LOGICAL_PROCESSOR_INFORMATION[len];
IntPtr Item = Ptr;
for (int i = 0; i < len; i++)
{
Buffer[i] = (SYSTEM_LOGICAL_PROCESSOR_INFORMATION)Marshal.PtrToStructure(Item, typeof(SYSTEM_LOGICAL_PROCESSOR_INFORMATION));
Item += size;
}
return Buffer;
}
}
finally
{
Marshal.FreeHGlobal(Ptr);
}
}
return null;
}
}

Understanding C++ DLL mapping to C# with unions

I have the following scenario. I have a struct with a union in a DLL written in C++ and I need to map this structure to C# code. Please, have in mind that I cannot change the code in the DLL. Here is a sample code to illustrate what is happening:
typedef struct {
int type;
union {
int val;
double val2;
char *name;
};
} BIZARRE;
__declspec(dllexport) void changeBizarre(BIZARRE *bizarre, int type, const char *v)
{
bizarre->type = type;
if(type == 0) {
bizarre->val = 5;
} else if(type == 1) {
bizarre->val2 = 2.0;
} else if(type == 2) {
strncpy(bizarre->name, "test", strlen("test"));
} else if(type == 3 && strcmp(v, "test") == 0) {
bizarre->val = 10;
}
}
And in the C# code, I've done the following:
[StructLayout(LayoutKind.Explicit)]
public unsafe struct BIZARRE
{
[FieldOffset(0)]
public int type;
[FieldOffset(8)]
public int val;
[FieldOffset(8)]
public double val2;
[FieldOffset(8)]
public char *name;
}
[DllImport("proj.dll", CallingConvention = CallingConvention.Cdecl)]
public unsafe static extern void changeBizarre(ref BIZARRE bizarre, int type, char *name);
unsafe static void Main()
{
char[] v = "test".ToCharArray();
bizarre.type = 0;
bizarre.val = 0;
bizarre.val2 = 0.0;
fixed (char* address = v)
{
bizarre.name = address;
changeBizarre(ref bizarre, 3, &bizarre.name);
Console.WriteLine("{0}", bizarre.val);
}
Console.ReadKey();
}
Now, if you run this code by passing type = 2, and try to print the bizarre.name, it will appear junk characters and if you pass type = 3, apparently the DLL is not being able to get the content pointed by bizarre.name, I do believe these two behaviors have the same cause, but I don't know what is it.

The char array is not a real char array in C, in fact C# will use wchar_t type (16 bit char).
Change your C code using wchar_t stuff:
typedef struct {
int type;
union {
int val;
double val2;
wchar_t *name;
};
} BIZARRE;
__declspec(dllexport) void changeBizarre(BIZARRE *bizarre, int type, const wchar_t *v)
{
bizarre->type = type;
if (type == 0) {
bizarre->val = 5;
}
else if (type == 1) {
bizarre->val2 = 2.0;
}
else if (type == 2) {
wcsncpy(bizarre->name, L"test", wcslen("test"));
}
else if (type == 3 && wcscmp(v, L"test") == 0) {
bizarre->val = 10;
}
}
Or
Change your C# code using the byte type instead of char (as mentioned #Hans Passant) :
[StructLayout(LayoutKind.Explicit)]
public unsafe struct BIZARRE
{
[FieldOffset(0)]
public int type;
[FieldOffset(8)]
public int val;
[FieldOffset(8)]
public double val2;
[FieldOffset(8)]
public byte* name;
}
class Program
{
[DllImport("UnionMapping_dll.dll", CallingConvention = CallingConvention.Cdecl)]
public unsafe static extern void changeBizarre(ref BIZARRE bizarre, int type, byte* name);
static unsafe void Main(string[] args)
{
BIZARRE bizarre;
byte[] v = Encoding.ASCII.GetBytes("test");
bizarre.type = 0;
bizarre.val = 0;
bizarre.val2 = 0.0;
fixed (byte* address = v)
{
bizarre.name = address;
changeBizarre(ref bizarre, 3, bizarre.name);
Console.WriteLine("{0}", bizarre.val);
}
Console.ReadKey();
}
}

The C++ code has a union within a struct - your C# equivalent for interop purposes should reflect that:
using System.Runtime.InteropServices;
public struct BIZARRE
{
public int type;
[StructLayout(LayoutKind.Explicit)]
public struct AnonymousStruct
{
[FieldOffset(0)]
public int val;
[FieldOffset(0)]
public double val2;
[FieldOffset(0)]
public string name;
}
}

Unpack cab file in memory

I am uploading a cab file from a web form, and want to unpack it in memory. I've tried tackling the issue with a CabInfo file, but without success. I do know how to unpack a cab file to my local disk, but do not know how to apply this in memory.
Any assistance would be appreciated.

If using another library is possible, take a look at this. The description clearly states the library will allow you to extract into memoty.
WIX is an open source project. You could always ask for this feature, ask for a better solution on their forum or simply modify the code for your need.
Vadim

Your initial question seems to indicate that you are allowed to use the Microsoft.Deployment.Compression DLL. If true, the code below should get you close to what you want:
CabEngine engine = new CabEngine();
foreach (ArchiveFileInfo archiveFileInfo in engine.GetFileInfo(fileStream))
{
Stream stream = engine.Unpack(fileStream, archiveFileInfo.Name);
byte[] buffer = new byte[stream.Length];
//The (int) below is a dirty trick for demonstration purposes only;
//re-work for production code
stream.Read(buffer, 0, (int)stream.Length);
}
Later answers from you seem to indicate that you are not allowed to use 3rd party DLLs in which case you will want to use the FDI* API. This link has some code that you will be able to modify from using hard-coded paths to (memory) streams: Extract .cab file in C#

There is a ready to use project that you can download: Cabinet File (*.CAB) Compression and Extraction
According to version history since Sep 2008 "..you can extract directly to memory."

Here is a utility class that should work in memory (it supports x86 or x64 compilation). Here is out you would use it, if we suppose a .CAB file has been uploaded into ASP.NET using the standard upload protocol:
using (CabFile file = new CabFile(HttpContext.Current.Request.Files[0].InputStream))
{
file.EntryExtract += CabEntryExtract;
file.ExtractEntries();
}
static void CabEntryExtract(object sender, CabEntryExtractEventArgs e)
{
// e.Entry.Name contains the entry name
// e.Entry.Data contains a byte[] with the entry data
// e.Entry.LastWriteTime contains the entry last write time
// e.Entry.Size contains the entry uncompressed size
}
And here is the utility and associated classes:
public sealed class CabFile : IDisposable
{
private IntPtr _hfdi;
private ERF _erf;
private GCHandle _erfHandle;
private byte[] _data;
private Dictionary<IntPtr, object> _handles = new Dictionary<IntPtr, object>();
private MemoryStream _currentEntryData;
private FNALLOC _alloc;
private FNCLOSE _close;
private FNFREE _free;
private FNOPEN _open;
private FNREAD _read;
private FNWRITE _write;
private FNSEEK _seek;
private FNFDINOTIFY _extract;
public event EventHandler<CabEntryExtractEventArgs> EntryExtract;
public CabFile(string filePath)
: this(GetStream(filePath))
{
}
private static Stream GetStream(string filePath)
{
if (filePath == null)
throw new ArgumentNullException("filePath");
return new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read);
}
public CabFile(Stream stream)
{
if (stream == null)
throw new ArgumentNullException("stream");
using (MemoryStream data = new MemoryStream())
{
stream.CopyTo(data);
_data = data.ToArray();
}
_erf = new ERF();
_alloc = new FNALLOC(FnAlloc);
_free = new FNFREE(FnFree);
_close = new FNCLOSE(FnClose);
_open = new FNOPEN(FnOpen);
_read = new FNREAD(FnRead);
_write = new FNWRITE(FnWrite);
_seek = new FNSEEK(FnSeek);
_extract = new FNFDINOTIFY(FnNotifyExtract);
_erfHandle = GCHandle.Alloc(_erf, GCHandleType.Pinned);
_hfdi = FDICreate(
Marshal.GetFunctionPointerForDelegate(_alloc),
Marshal.GetFunctionPointerForDelegate(_free),
Marshal.GetFunctionPointerForDelegate(_open),
Marshal.GetFunctionPointerForDelegate(_read),
Marshal.GetFunctionPointerForDelegate(_write),
Marshal.GetFunctionPointerForDelegate(_close),
Marshal.GetFunctionPointerForDelegate(_seek)
, -1, _erfHandle.AddrOfPinnedObject());
}
public void ExtractEntries()
{
FDICopy(_hfdi, string.Empty, string.Empty, 0, Marshal.GetFunctionPointerForDelegate(_extract), IntPtr.Zero, IntPtr.Zero);
}
public void Dispose()
{
if (_hfdi != IntPtr.Zero)
{
FDIDestroy(_hfdi);
_hfdi = IntPtr.Zero;
}
_erfHandle.Free();
}
private void OnEntryExtract(CabEntry entry)
{
EventHandler<CabEntryExtractEventArgs> handler = EntryExtract;
if (handler != null)
{
handler(this, new CabEntryExtractEventArgs(entry));
}
}
private IntPtr FnAlloc(int cb)
{
return Marshal.AllocHGlobal(cb);
}
private void FnFree(IntPtr pv)
{
Marshal.FreeHGlobal(pv);
}
private IntPtr FnOpen(string pszFile, int oflag, int pmode)
{
// only used for reading archive
IntPtr h = new IntPtr(_handles.Count + 1);
_handles.Add(h, 0);
return h;
}
private int FnRead(IntPtr hf, byte[] pv, int cb)
{
// only used for reading archive
int pos = (int)_handles[hf];
int left = _data.Length - pos;
int read = Math.Min(left, cb);
if (read > 0)
{
Array.Copy(_data, pos, pv, 0, read);
_handles[hf] = pos + read;
}
return read;
}
private int FnWrite(IntPtr hf, byte[] pv, int cb)
{
// only used for writing entries
_currentEntryData.Write(pv, 0, cb);
return cb;
}
private int FnClose(IntPtr hf)
{
object o = _handles[hf];
CabEntry entry = o as CabEntry;
if (entry != null)
{
entry.Data = _currentEntryData.ToArray();
_currentEntryData.Dispose();
}
_handles.Remove(hf);
return 0;
}
private int FnSeek(IntPtr hf, int dist, SeekOrigin seektype)
{
// only used for seeking archive
int pos;
switch (seektype)
{
case SeekOrigin.Begin:
pos = dist;
break;
case SeekOrigin.Current:
pos = (int)_handles[hf] + dist;
break;
//case SeekOrigin.End:
default:
pos = _data.Length + dist;
break;
}
_handles[hf] = pos;
return pos;
}
private IntPtr FnNotifyExtract(FDINOTIFICATIONTYPE fdint, FDINOTIFICATION fdin)
{
CabEntry entry;
switch (fdint)
{
case FDINOTIFICATIONTYPE.COPY_FILE:
entry = new CabEntry(fdin);
entry._handle = new IntPtr(_handles.Count + 1);
_handles.Add(entry._handle, entry);
_currentEntryData = new MemoryStream();
return entry._handle;
case FDINOTIFICATIONTYPE.CLOSE_FILE_INFO:
entry = (CabEntry)_handles[fdin.hf];
FnClose(fdin.hf);
OnEntryExtract(entry);
return new IntPtr(1);
default:
return IntPtr.Zero;
}
}
private enum FDINOTIFICATIONTYPE
{
CABINET_INFO = 0,
PARTIAL_FILE = 1,
COPY_FILE = 2,
CLOSE_FILE_INFO = 3,
NEXT_CABINET = 4,
ENUMERATE = 5,
}
[StructLayout(LayoutKind.Sequential)]
private struct ERF
{
public int erfOper;
public int erfType;
public int fError;
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
internal class FDINOTIFICATION
{
public int cb;
public IntPtr psz1;
public IntPtr psz2;
public IntPtr psz3;
public IntPtr pv;
public IntPtr hf;
public ushort date;
public ushort time;
public ushort attribs;
public ushort setID;
public ushort iCabinet;
public ushort iFolder;
public int fdie;
}
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate IntPtr FNALLOC(int cb);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void FNFREE(IntPtr pv);
[UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private delegate IntPtr FNOPEN(string pszFile, int oflag, int pmode);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate int FNREAD(IntPtr hf, [Out, MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 2)] byte[] pv, int cb);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate int FNWRITE(IntPtr hf, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 2)] byte[] pv, int cb);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate int FNCLOSE(IntPtr hf);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate int FNSEEK(IntPtr hf, int dist, SeekOrigin seektype);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate IntPtr FNFDINOTIFY(FDINOTIFICATIONTYPE fdint, FDINOTIFICATION pfdin);
[DllImport("cabinet.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern IntPtr FDICreate(IntPtr pfnalloc, IntPtr pfnfree, IntPtr pfnopen, IntPtr pfnread, IntPtr pfnwriter, IntPtr pfnclose, IntPtr pfnseek, int cpuType, IntPtr perf);
[DllImport("cabinet.dll", CallingConvention = CallingConvention.Cdecl)]
private static extern IntPtr FDIDestroy(IntPtr hdfi);
[DllImport("cabinet.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern IntPtr FDICopy(IntPtr hdfi, string pszCabinet, string pszCabPath, int flags, IntPtr fnNotify, IntPtr fnDecrypt, IntPtr userData);
}
public sealed class CabEntry
{
internal IntPtr _handle;
internal CabEntry(CabFile.FDINOTIFICATION fdin)
{
Name = Marshal.PtrToStringAnsi(fdin.psz1);
Size = fdin.cb;
LastWriteTime = new DateTime(1980 + GetMask(fdin.date, 9, 15), GetMask(fdin.date, 5, 8), GetMask(fdin.date, 0, 4),
GetMask(fdin.time, 11, 15), GetMask(fdin.time, 5, 10), 2 * GetMask(fdin.time, 0, 4));
}
private static int GetMask(int value, byte startByte, byte endByte)
{
int final = 0;
int v = 1;
for (byte b = startByte; b <= endByte; b++)
{
if ((value & (1 << b)) != 0)
{
final += v;
}
v = v * 2;
}
return final;
}
public string Name { get; private set; }
public int Size { get; private set; }
public DateTime LastWriteTime { get; private set; }
public byte[] Data { get; internal set; }
}
public sealed class CabEntryExtractEventArgs : EventArgs
{
public CabEntryExtractEventArgs(CabEntry entry)
{
if (entry == null)
throw new ArgumentNullException("entry");
Entry = entry;
}
public CabEntry Entry { get; private set; }
}
NOTE: this code allocates big byte[] chunks so it could be optimized to use things like ChunkedMemoryStream (available for example in this library: CodeFluent Runtime Client) instead of byte[] to avoid impacting the LOH (Large Object Heap) too much.

Is there an alternative for StructLayout "Pack" attribute in Compact Framework?

I would like to do the following:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct SomeStruct
{
public byte SomeByte;
public int SomeInt;
public short SomeShort;
public byte SomeByte2;
}
Is there an alternative since Pack is not supported in the compact framework?
Update: Explicitly setting up the structure and giving FieldOffset for each does not work either as it does not affect how the struct is packed
Update2: If you try the following, the CF program wont even run because of how the structure is packed:
[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
[FieldOffset(0)]
public byte SomeByte;
[FieldOffset(1)]
public int SomeInt;
[FieldOffset(5)]
public short SomeShort;
[FieldOffset(7)]
public byte SomeByte2;
}
I know it seems hard to believe, but if you try it you will see. Add it to a CF project and try to run it and you will get a TypeLoadException. Changing the offsets to 0,4,8,10 respectively and it will work (but the size ends up being 12).
I was hoping maybe someone had a solution using reflection maybe to marshal the size of each of the field types individually (something involving recursion to handle structs within structs or arrays of types).

This probably isn't the type of answer you're looking for, but I'll post it anyway for the hell of it:
public struct SomeStruct
{
public byte SomeByte;
public int SomeInt;
public short SomeShort;
public byte SomeByte2;
public byte[] APIStruct
{
get
{
byte[] output = new byte[8];
output[0] = this.SomeByte;
Array.Copy(BitConverter.GetBytes(this.SomeInt), 0,
output, 1, 4);
Array.Copy(BitConverter.GetBytes(this.SomeShort), 0,
output, 5, 2);
output[7] = this.SomeByte2;
return output;
}
set
{
byte[] input = value;
this.SomeByte = input[0];
this.SomeInt = BitConverter.ToInt32(input, 1);
this.SomeShort = BitConverter.ToInt16(input, 5);
this.SomeByte2 = input[7];
}
}
}
Basically it does the packing/unpacking itself in the APIStruct property.

The simplest method of dealing with this type of problem is along the same lines as you might for a bit field, simply pack your data into a private member (or members if it is large) of the appropriate data type and then present public properties that unpack the data for you. The unpacking operations are extremely fast and will have little impact on performance. For your particular type the following is probably what you want:
public struct SomeStruct
{
private long data;
public byte SomeByte { get { return (byte)(data & 0x0FF); } }
public int SomeInt { get { return (int)((data & 0xFFFFFFFF00) << 8); } }
public short SomeShort { get { return (short)((data & 0xFFFF0000000000) << 40); } }
public byte SomeByte2 { get { return (byte)((data & unchecked((long)0xFF00000000000000)) << 56); } }
}
For some structures even this method is not workable due to the unfortunate way a structure has been defined. In those cases you will generally have to use a byte array as a blob of data from which the elements can be unpacked.
EDIT: To expand on what I mean about structs that can't be handled using this simple method. When you can't do simple packing/unpacking like this you need to manually marshal the irregular struct . This can be done using manual methods at the point you call the pInvoked API or by using a custom marshaler. The following is an example of a custom marhsaler that can be easily adapted to on the spot manual marshaling.
using System.Runtime.InteropServices;
using System.Threading;
public class Sample
{
[DllImport("sample.dll")]
public static extern void TestDataMethod([MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(TestDataMarshaler))] TestDataStruct pData);
}
public class TestDataStruct
{
public byte data1;
public int data2;
public byte[] data3 = new byte[7];
public long data4;
public byte data5;
}
public class TestDataMarshaler : ICustomMarshaler
{
//thread static since this could be called on
//multiple threads at the same time.
[ThreadStatic()]
private static TestDataStruct m_MarshaledInstance;
private static ICustomMarshaler m_Instance = new TestDataMarshaler();
public static ICustomFormatter GetInstance(string cookie)
{
return m_Instance;
}
#region ICustomMarshaler Members
public void CleanUpManagedData(object ManagedObj)
{
//nothing to do.
}
public void CleanUpNativeData(IntPtr pNativeData)
{
Marshal.FreeHGlobal(pNativeData);
}
public int GetNativeDataSize()
{
return 21;
}
public IntPtr MarshalManagedToNative(object ManagedObj)
{
m_MarshaledInstance = (TestDataStruct)ManagedObj;
IntPtr nativeData = Marshal.AllocHGlobal(GetNativeDataSize());
if (m_MarshaledInstance != null)
{
unsafe //unsafe is simpler but can easily be done without unsafe if necessary
{
byte* pData = (byte*)nativeData;
*pData = m_MarshaledInstance.data1;
*(int*)(pData + 1) = m_MarshaledInstance.data2;
Marshal.Copy(m_MarshaledInstance.data3, 0, (IntPtr)(pData + 5), 7);
*(long*)(pData + 12) = m_MarshaledInstance.data4;
*(pData + 20) = m_MarshaledInstance.data5;
}
}
return nativeData;
}
public object MarshalNativeToManaged(IntPtr pNativeData)
{
TestDataStruct data = m_MarshaledInstance;
m_MarshaledInstance = null; //clear out TLS for next call.
if (data == null) data = new TestDataStruct(); //if no in object then return a new one
unsafe //unsafe is simpler but can easily be done without unsafe if necessary
{
byte* pData = (byte*)pNativeData;
data.data1 = *pData;
data.data2 = *(int*)(pData + 1);
Marshal.Copy((IntPtr)(pData + 5), data.data3, 0, 7);
data.data4 = *(long*)(pData + 12);
data.data5 = *(pData + 20);
}
return data;
}
#endregion
}
In the case of arrays of these structs you can't use custom marshaling unless the array size is fixed but it is relatively easy to manually marshal the array data as a whole using the same techniques.

Do you absolutely require that specific layout or is it acceptable to simply make the size 8?
I ask this because the lay out as follows
[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
[FieldOffset(0)]
public byte SomeByte;
[FieldOffset(1)]
public int SomeInt;
[FieldOffset(5)]
public short SomeShort;
[FieldOffset(7)]
public byte SomeByte2;
}
Has non word aligned fields which may be what is causing your problem.
If you can 'rearrange' things then this might work for you:
[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
[FieldOffset(0)]
public byte SomeByte;
[FieldOffset(1)]
public byte SomeByte2;
[FieldOffset(2)]
public short SomeShort;
[FieldOffset(4)]
public int SomeInt;
}
When I test with this on the emulator it works fine.
Obviously unless you are willing to allow the rearrangement there's nothing you can do.
This answer and this old article would strongly indicate that you must at a minimum align your structs on multiples of their size (I tried with an int aligned on offset 2 and this also triggered the error)
Given your need to interoperate with externally defined data then the following is likely your easiest solution:
[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
[FieldOffset(0)] private byte b0;
[FieldOffset(1)] private byte b1;
[FieldOffset(2)] private byte b2;
[FieldOffset(3)] private byte b3;
[FieldOffset(4)] private byte b4;
[FieldOffset(5)] private byte b5;
[FieldOffset(6)] private byte b6;
[FieldOffset(7)] private byte b7;
// not thread safe - alter accordingly if that is a requirement
private readonly static byte[] scratch = new byte[4];
public byte SomeByte
{
get { return b0; }
set { b0 = value; }
}
public int SomeInt
{
get
{
// get the right endianess for your system this is just an example!
scratch[0] = b1;
scratch[1] = b2;
scratch[2] = b3;
scratch[3] = b4;
return BitConverter.ToInt32(scratch, 0);
}
}
public short SomeShort
{
get
{
// get the right endianess for your system this is just an example!
scratch[0] = b5;
scratch[1] = b6;
return BitConverter.ToInt16(scratch, 0);
}
}
public byte SomeByte2
{
get { return b7; }
set { b7 = value; }
}
}

You need to post a more relevant example. Setting packing on that struct would have no effect anyway.
My bet is that you need to use LaoutKind.Explicit and then give the offsets for each member. It's way better than messing with the packing anyway, because it's way more obvious to someone looking at the code that the original developer explicitly meant for things to be unaligned.
Something along these lines:
[StructLayout(LayoutKind.Explicit)]
struct Foo
{
[FieldOffset(0)]
byte a;
[FieldOffset(1)]
uint b;
}

I think one should take Stephen Martin's answer, make it accept a T, and use reflection to generically implement the MarshalManagedToNative and MarshalNativeToManaged methods. Then, you'll have a custom packed struct marshaler that will work for any type of struct.
Here's the code:
using System;
using System.Threading;
using System.Reflection;
using System.Runtime.InteropServices;
namespace System.Runtime.InteropServices
{
public class PinnedObject : IDisposable
{
private GCHandle gcHandle = new GCHandle();
public PinnedObject(object o)
{
gcHandle = GCHandle.Alloc(o, GCHandleType.Pinned);
}
public unsafe static implicit operator byte*(PinnedObject po)
{
return (byte*)po.gcHandle.AddrOfPinnedObject();
}
#region IDisposable Members
public void Dispose()
{
if (gcHandle.IsAllocated)
{
gcHandle.Free();
}
}
#endregion
}
public class PackedStructMarshaler<T> : ICustomMarshaler where T : struct
{
private static ICustomMarshaler m_instance = new PackedStructMarshaler<T>();
public static ICustomMarshaler GetInstance()
{
return m_instance;
}
private void ForEachField(Action<FieldInfo> action)
{
foreach (var fi in typeof(T).GetFields(BindingFlags.Public | BindingFlags.NonPublic))
{
// System.Diagnostics.Debug.Assert(fi.IsValueType);
action(fi);
}
}
private unsafe void MemCpy(byte* dst, byte* src, int numBytes)
{
for (int i = 0; i < numBytes; i++)
{
dst[i] = src[i];
}
}
#region ICustomMarshaler Members
public void CleanUpManagedData(object ManagedObj)
{
}
public void CleanUpNativeData(IntPtr pNativeData)
{
Marshal.FreeHGlobal(pNativeData);
}
public int GetNativeDataSize()
{
unsafe
{
int ret = 0;
ForEachField(
(FieldInfo fi) =>
{
Type ft = fi.FieldType;
ret += Marshal.SizeOf(ft);
});
return ret;
}
}
private object m_marshaledObj = null;
public unsafe IntPtr MarshalManagedToNative(object obj)
{
IntPtr nativeData = (IntPtr)0;
if (obj != null)
{
if (m_marshaledObj != null)
throw new ApplicationException("This instance has already marshaled a managed type");
m_marshaledObj = obj;
nativeData = Marshal.AllocHGlobal(GetNativeDataSize());
byte* pData = (byte*)nativeData;
int offset = 0;
ForEachField(
(FieldInfo fi) =>
{
int size = Marshal.SizeOf(fi.FieldType);
using (PinnedObject po = new PinnedObject(fi.GetValue(obj)))
{
MemCpy(pData + offset, po, size);
}
offset += size;
});
}
return nativeData;
}
public object MarshalNativeToManaged(IntPtr pNativeData)
{
if (m_marshaledObj != null)
m_marshaledObj = null;
unsafe
{
byte* pData = (byte*)pNativeData;
int offset = 0;
object res = new T();
ForEachField(
(FieldInfo fi) =>
{
int size = Marshal.SizeOf(fi.FieldType);
fi.SetValue(res, (object)(*((byte*)(pData + offset))));
offset += size;
});
return res;
}
}
#endregion
}
}

LayoutKind.Explicit would be your best bet for defining a specific memory layout. However, do not use LayoutKind.Explicit for structures that contain pointer-sized values such as true pointers, operating system handles or IntPtrs; this is just asking for mysterious trouble at runtime on random platforms.
In particular, LayoutKind.Explicit is a poor substitute for anonymous unions. If your target structure contains an anonymous union, convert it to a named union; you can safely represent a named union as a struct with LayoutKind.Explicit where all offsets are 0.

LayoutKind.Explicit and FieldOffsetAttribute will allow you to do anything you could do with the Pack property. These explicit layout attributes allow you to specify the exact byte position of each field in the struct (relative to the beginning of the struct's range of memory). The Pack property is used by the runtime to help determine the exact position of each field when using a sequential layout. The pack property has no other effect, so using explicit layout allows you to emulate the exact same behavior, albeit a bit more verbosely. If you don't think this solves your problem, perhaps you could post a bit more information about what you're trying to do or why you think you need to use the Pack property.
Edit: I just noticed the additional comment about trying to get the entire structure's size to 8 bytes. Have you tried using the StructLayoutAttribute.Size property? Unlike Pack, it is available in the Compact Framework.