Develop Reference

Porting binary file reading C++ code to C#

I am trying to port some code from our C++ application to ensure that the licence files are compatible with both applications.
I've tried marking the functions as external, however that opened a whole other can of worms that I wouldn't like to delve into considering this is my last week at my current employer.
The C++ code I'd like to port is as follows:
std::fstream licenceFile(filePath, std::ios::in | std::ios::binary);
licenceFile.read((char *) &this->m_FileData, sizeof(this->m_FileData));
m_FileData is a struct and is as follows:
struct LicManKeyFileData
{
BYTE m_EncryptedKey[255];
WCHAR m_MacAddress[33];
WCHAR m_PreviousMacLicense[33];
WCHAR m_SoftwareName[16];
WCHAR m_ClientName[65];
BYTE m_Version;
BYTE m_EncryptionLength;
time_t m_LicenseTime;
};
I have tried several methods of replicating this in C#, firstly reading the file member by member, for example:
BinaryReader reader = new BinaryReader(licenceFileStream, Encoding.BigEndianUnicode);
licenceFileData.EncryptedKey = reader.ReadBytes(licenceFileData.EncryptedKey.Length);
byte[] mac = new byte[sizeof(char) * licenceFileData.MacAddress.Length];
mac = reader.ReadBytes(mac.Length);
Something strange I noticed about this method is that a lot of the values were being set as 204 seemingly arbitrarily, when they are read correctly in the C++ application, so I've had to add this:
if (mac[0] == 204)
mac[0] = 0;
Which makes no sense to me, it's not an alignment issue, reading from a byte before or after that gives totally garbage values (chinese characters) and this check and assigning 0 to the first index allows me to get the proper value.
Another method I've tried was found on here on StackOverflow, and is as follows:
public static T ReadStruct<T>(this BinaryReader reader) where T : struct
{
Byte[] buffer = new Byte[Marshal.SizeOf(typeof(T))];
reader.Read(buffer, 0, buffer.Length);
GCHandle handle = default(GCHandle);
try
{
handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
return (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
}
finally
{
if (handle.IsAllocated)
handle.Free();
}
}
Oddly, this method also returns some values as 204 where the C++ application reads them as the correct value.
The struct I'm using in C# looks like this:
unsafe struct LicenceFileDataS
{
internal fixed byte m_EncryptedKey[255];
internal fixed char m_MacAddress[33];
internal fixed char m_PreviousMacLicense[33];
internal fixed char m_SoftwareName[16];
internal fixed char m_ClientName[65];
internal byte m_Version;
internal byte m_EncryptionLength;
internal long m_LicenseTime;
}
Does anyone know a better method for porting this code or how I can fix the values that are being read as 204? It works perfectly in C++. My apologies for the wall of text.

How can I marshal a Delphi short string using p/invoke?

I have a problem with a variable type from a dll which i am importing in C#. It is written in object oriented pascal and it says it is developed with the Delphi Development Tool.
On the manual of the library it says that shortstring is a packed array of bytes. The first byte is the length and the following are exaclty 255 bytes with the string data that i need.
So after importing the dll in C# i wrote:
[return: MarshalAs(UnmanagedType.LPArray)]
Then called the function from the dll:
public static extern byte[] xxxx();
But i am getting the following error:
Cannot marshal 'return value': invalid managed/unmanaged type combination
On the other hand i tried the following:
[return: MarshalAs(UnmanagedType.SafeArray)]
This time i get the error:
SafeArray of rank 18727 has been passed to a method expecting an array of rank 1
Can you please tell me what i am doing wrong, and firstly is correct like this to get a shortstring from a compiled library?
Regards

I think the cleanest way to marshal a Delphi short string is to wrap it in a struct.
[StructLayout(LayoutKind.Sequential)]
public struct DelphiShortString
{
private byte length;
[MarshalAs(UnmanagedType.ByValArray, SizeConst=255)]
private byte[] payload;
public string Value
{
get
{
return Encoding.Default.GetString(payload, 0, length);
}
set
{
length = (byte)Math.Min(value.Length, 255);
payload = Encoding.Default.GetBytes(value.Substring(0, length));
}
}
}
This type is not blittable and so cannot be used as a function return value. If you have control of the Delphi code, then you can ensure that you don't use short strings as function return types. However, I suspect that you don't have control of it. In which case you'd need a blittable version of the struct. Which would look like this:
[StructLayout(LayoutKind.Sequential)]
public unsafe struct DelphiShortString
{
private byte length;
private fixed byte payload[255];
public string Value
{
get
{
{
byte[] bytes = new byte[length];
fixed (byte* ptr = payload)
{
Marshal.Copy((IntPtr)ptr, bytes, 0, length);
}
return Encoding.Default.GetString(bytes, 0, length);
}
}
set
{
byte[] bytes = Encoding.Default.GetBytes(value);
length = (byte)Math.Min(bytes.Length, 255);
fixed (byte* ptr = payload)
{
Marshal.Copy(bytes, 0, (IntPtr)ptr, length);
}
}
}
}
It is indicative of poor design for a DLL to export a Delphi short string. That suggests that the author of the library has not designed it to be compatible with other compilers. Which in turn suggests that the functions may well use the default Delphi calling convention. Which is register and is not supported by Microsoft tools. If my hunch is right, then this DLL cannot be consumed directly. You would need to write an adapter DLL that exposed a more interop friendly interface.

How do I call this c function in c# (unmarshalling return struct)?

I want to use c# interop to call a function from a dll written in c. I have the header files.
Take a look at this:
enum CTMBeginTransactionError {
CTM_BEGIN_TRX_SUCCESS = 0,
CTM_BEGIN_TRX_ERROR_ALREADY_IN_PROGRESS,
CTM_BEGIN_TRX_ERROR_NOT_CONNECTED
};
#pragma pack(push)
#pragma pack(1)
struct CTMBeginTransactionResult {
char * szTransactionID;
enum CTMBeginTransactionError error;
};
struct CTMBeginTransactionResult ctm_begin_customer_transaction(const char * szTransactionID);
How do I call ctm_begin_customer_transaction from c#. The const char * mapps well to string, but despite various attempts (looking at stackoverflow and other sites), I fail to marshal the return structure. If I define the function to return IntPtr it works ok...
Edit
I changed the return type to IntPtr and use:
CTMBeginTransactionResult structure = (CTMBeginTransactionResult)Marshal.PtrToStructure(ptr, typeof(CTMBeginTransactionResult));
but it throws AccessViolationException
I also tried:
IntPtr ptr = Transactions.ctm_begin_customer_transaction("");
int size = 50;
byte[] byteArray = new byte[size];
Marshal.Copy(ptr, byteArray, 0, size);
string stringData = Encoding.ASCII.GetString(byteArray);
stringData == "70e3589b-2de0-4d1e-978d-55e22225be95\0\"\0\0\a\0\0\b\b?" at this point. The "70e3589b-2de0-4d1e-978d-55e22225be95" is the szTransactionID from the struct. Where is the Enum? Is it the next byte?

There's a memory management problem hidden in this struct. Who owns the C string pointer? The pinvoke marshaller will always assume that the caller owns it so it will try to release the string. And passes the pointer to CoTaskMemFree(), same function as the one called by Marshal.FreeCoTaskMem(). These functions use the COM memory allocator, the universal interop memory manager in Windows.
This rarely comes to a good end, C code does not typically use that allocator unless the programmer designed his code with interop in mind. In which case he'd never have used a struct as a return value, interop always works much less trouble-free when the caller supplies buffers.
So you cannot afford to let the marshaller do its normal duty. You must declare the return value type as IntPtr so it doesn't try to release the string. And you must marshal it yourself with Marshal.PtrToStructure().
That however still leaves the question unanswered, who owns the string? There is nothing you can do to release the string buffer, you don't have access to the allocator used in the C code. The only hope you have is that the string wasn't actually allocated on the heap. That's possible, the C program might be using string literals. You need to verify that guess. Call the function a billion times in a test program. If that doesn't explode the program then you're good. If not then only C++/CLI can solve your problem. Given the nature of the string, a "transaction ID" ought to change a lot, I'd say you do have a problem.

I hate to answer my own question, but I found the solution to marshal the resulting struct. The struct is 8 bytes long (4 bytes for the char * and 4 bytes for enum). Marshalling the string does not work automatically, but the following works:
// Native (unmanaged)
public enum CTMBeginTransactionError
{
CTM_BEGIN_TRX_SUCCESS = 0,
CTM_BEGIN_TRX_ERROR_ALREADY_IN_PROGRESS,
CTM_BEGIN_TRX_ERROR_NOT_CONNECTED
};
// Native (unmanaged)
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
internal struct CTMBeginTransactionResult
{
public IntPtr szTransactionID;
public CTMBeginTransactionError error;
};
// Managed wrapper around native struct
public class BeginTransactionResult
{
public string TransactionID;
public CTMBeginTransactionError Error;
internal BeginTransactionResult(CTMBeginTransactionResult nativeStruct)
{
// Manually marshal the string
if (nativeStruct.szTransactionID == IntPtr.Zero) this.TransactionID = "";
else this.TransactionID = Marshal.PtrToStringAnsi(nativeStruct.szTransactionID);
this.Error = nativeStruct.error;
}
}
[DllImport("libctmclient-0.dll")]
internal static extern CTMBeginTransactionResult ctm_begin_customer_transaction(string ptr);
public static BeginTransactionResult BeginCustomerTransaction(string transactionId)
{
CTMBeginTransactionResult nativeResult = Transactions.ctm_begin_customer_transaction(transactionId);
return new BeginTransactionResult(nativeResult);
}
The code works, but I still need to investigate, if calling the unmanaged code results in memory leaks.

C# calling C function that returns struct with fixed size char array

So, there have been many variants of this question, and after looking at several I still can't figure it out.
This is the C code:
typedef struct
{
unsigned long Identifier;
char Name[128];
} Frame;
Frame GetFrame(int index);
This is the C# code:
struct Frame
{
public ulong Identifier;
[MarshalAs(UnmanagedType.ByValArray, ArraySubType = UnmanagedType.I1, SizeConst = 128)]
public char[] Name;
}
[DllImport("XNETDB.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern Frame GetFrame(int index);
This is the last attempt I tried in C#, and it seems pretty logical, but I get the error "Method's signature is not PInvoke compatible." So, I'm kind of lost on what to try next. Any help is appreciated.
Thanks,
Kevin
Updated Kevin added this as an edit to my answer
I should instead change my C code:
void GetFrame(int index, Frame * f);
and use instead for C#:
struct Frame
{
public uint Identifier;
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 128)]
public string Name;
}
[DllImport("XNETDB.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern void GetFrame(int index, ref Frame f);

There are two problems with the PInvoke signature that you've chosen.
The first is easy to fix. You have a mistranslation of unsigned long. In C an unsigned long is typically only 4 bytes. You chose the C# type long which is 8 bytes. Changing the C# code to use uint will fix this.
The second is a bit harder. As Tergiver pointed out the CLR Marshaller only supports a struct in the return position if it's blittable. Blittable is a fancy way of saying that it has the exact same memory representation in native and managed code. The struct definition you've chosen isn't blittable because it has a nested array.
This can be worked around though if you remember that PInvoke is a very simple process. The CLR Marshaller really just needs you to answer 2 questions with the signature of your types and pinvoke methods
How many bytes am I copying?
In which direction do they need to go?
In this case the number of bytes is sizeof(unsigned long) + 128 == 132. So all we need to do is build up a managed type that is blittable and has a size of 132 bytes. The easiest way to do this is to define a blob to handle the array portion
[StructLayout(LayoutKind.Sequential, Size = 128)]
struct Blob
{
// Intentionally left empty. It's just a blob
}
This is a struct with no members that will appear to the marshaller as having a size of 128 bytes (and as a bonus it's blittable!). Now we can easily define the Frame structure as a combination of an uint and this type
struct Frame
{
public int Identifier;
public Blob NameBlob;
...
}
Now we have a blittable type with a size the marshaller will see as 132 bytes. This means it will work just fine with the GetFrame signature you've defined
The only part left is giving you access to the actual char[] for the name. This is a bit tricky but can be solved with a bit of marshal magic.
public string GetName()
{
IntPtr ptr = IntPtr.Zero;
try
{
ptr = Marshal.AllocHGlobal(128);
Marshal.StructureToPtr(NameBlob, ptr, false);
return Marshal.PtrToStringAnsi(ptr, 128);
}
finally
{
if (ptr != IntPtr.Zero)
{
Marshal.FreeHGlobal(ptr);
}
}
}
Note: I can't comment on the calling convention portion because I'm unfamiliar with the GetFrame API but that's something i would definitely check on.

The problem is that the native function returns a non-blittable type as a return value.
http://msdn.microsoft.com/en-us/library/ef4c3t39.aspx
P/Invoke cannot have non-blittable types as a return value.
You cannot p/Invoke that method. [EDIT It is actually possible, see JaredPar's answer]
Returning 132 bytes by value is a bad idea. If this native code is yours, I would fix it. You can fix it by allocating the 132 bytes and returning a pointer. Then add a FreeFrame method to release that memory. Now it can be p/Invoked.
Alternately, you could change it to accept a pointer to the Frame memory that it will fill in.

Another option to JaredPar's is to utilize the C# fixed size buffer feature. This does however require you to turn on the setting to allow unsafe code, but it avoids having 2 structs.
class Program
{
private const int SIZE = 128;
unsafe public struct Frame
{
public uint Identifier;
public fixed byte Name[SIZE];
}
[DllImport("PinvokeTest2.DLL", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern Frame GetFrame(int index);
static unsafe string GetNameFromFrame(Frame frame)
{
//Option 1: Use if the string in the buffer is always null terminated
//return Marshal.PtrToStringAnsi(new IntPtr(frame.Name));
//Option 2: Use if the string might not be null terminated for any reason,
//like if were 128 non-null characters, or the buffer has corrupt data.
return Marshal.PtrToStringAnsi(new IntPtr(frame.Name), SIZE).Split('\0')[0];
}
static void Main()
{
Frame a = GetFrame(0);
Console.WriteLine(GetNameFromFrame(a));
}
}

converting a byte array from a native method to a managed structure

I have a c# .net 2.0 CF application that interfaces with a native DLL implementing a function like this:
struct NATIVE_METHOD_REPLY
{
int other_irrelevant_data;
int data_size;
void* data;
}
// reply_buffer will contain an array of NATIVE_METHOD_REPLY structures
// and their data.
//
// returns an error code
int Foo(NATIVE_METHOD_REPLY* reply_buffer, int reply_size);
I've implemented it in C# like this:
[StructLayout(LayoutKind.Sequential)]
internal struct NATIVE_METHOD_REPLY
{
public Int32 OtherIrrelevantData;
public Int16 DataSize;
public IntPtr DataPtr;
}
[DllImport("my_lib.dll", SetLastError = true)]
internal static extern Int32 Foo(byte[] replyBuffer, Int32 replySize);
public byte[] void Bar()
{
// data returned to the user. May be an arbitrary size.
byte[] result_buffer = new byte[256];
// data sent to Foo()
byte[] reply_buffer =
new byte[Marshal.SizeOf(typeof(NativeMethods.NATIVE_METHOD_REPLY)) +
result_buffer.Length];
NativeMethods.Foo(reply_buffer, reply_buffer.Length);
// is there a better way of doing this?
NativeMethods.NATIVE_METHOD_REPLY reply;
GCHandle pinned_reply = GCHandle.Alloc(reply_buffer,
GCHandleType.Pinned);
try
{
reply = (NativeMethods.NATIVE_METHOD_REPLY)Marshal.PtrToStructure(
pinned_reply.AddrOfPinnedObject(),
typeof(NativeMethods.NATIVE_METHOD_REPLY));
Marshal.Copy(reply.DataPtr, result_buffer, 0, reply.DataSize);
}
finally
{
pinned_reply.Free();
}
// bonus point*: is this okay to do after the Free() call?
int test = reply.OtherIrrelevantData;
return result_buffer;
}
While this works correctly, I would like to know if this is the most efficient / most correct way of implementing this function.
Is there some method converting a managed byte array to a managed structure that doesn't involve an intermediate native handle and a copy? For instance, in C++, I would just do this:
NATIVE_METHOD_REPLY* reply = reinterpret_cast< NATIVE_METHOD_REPLY* >( reply.DataPtr );
*For a bonus point, is it okay to use data in the structure after the native handle has been freed?
Thanks,
PaulH
Edit: Updated solution
[DllImport("my_lib.dll", SetLastError = true)]
internal static extern Int32 Foo(IntPtr replyBuffer, Int32 replySize);
public byte[] void Bar()
{
byte[] result_buffer = new byte[256];
int reply_buffer_len = Marshal.SizeOf(typeof(NativeMethods.NATIVE_METHOD_REPLY)) + result_buffer.Length;
IntPtr reply_buffer = Marshal.AllocCoTaskMem(reply_buffer_len);
NativeMethods.NATIVE_METHOD_REPLY reply;
try
{
NativeMethods.Foo(reply_buffer, reply_buffer_len);
reply = (NativeMethods.NATIVE_METHOD_REPLY)Marshal.PtrToStructure(
reply_buffer,
typeof(NativeMethods.NATIVE_METHOD_REPLY));
Marshal.Copy(reply.DataPtr, result_buffer, 0, reply.DataSize);
}
finally
{
Marshal.FreeCoTaskMem(reply_buffer);
}
return result_buffer;
}

The structure has a fixed size. There's no point in passing an array, just pass the structure:
[DllImport("my_lib.dll", SetLastError = true)]
internal static extern Int32 Foo(out NATIVE_METHOD_REPLY replyBuffer, Int32 replySize);
You do have a memory management problem. Who owns the pointer?
Okay, the structure is actually variable sized and the pointer points into the array. You need nother approach. Simply allocate a chunk of unmanaged memory up front instead of letting the P/Invoke marshaller copy the data into a managed array. Which is in fact a hard requirement since the garbage collector can move the array, invalidating the pointer. Call Marshal.CoTaskMemAlloc() to reserve the memory, you'll have to free it later. And change the first argument of the function to IntPtr (not out).
You'll find that marshaling the structure a lot easier too, no need to pin the memory. Don't forget to Marshal.FreeCoTaskMem() when you're done.

In C# under the full framework, you can marshal the array directly. See Default Marshaling for Arrays. I don't know what the limitations are on the Compact Framework.