Develop Reference

Porting binary file reading C++ code to C#

I am trying to port some code from our C++ application to ensure that the licence files are compatible with both applications.
I've tried marking the functions as external, however that opened a whole other can of worms that I wouldn't like to delve into considering this is my last week at my current employer.
The C++ code I'd like to port is as follows:
std::fstream licenceFile(filePath, std::ios::in | std::ios::binary);
licenceFile.read((char *) &this->m_FileData, sizeof(this->m_FileData));
m_FileData is a struct and is as follows:
struct LicManKeyFileData
{
BYTE m_EncryptedKey[255];
WCHAR m_MacAddress[33];
WCHAR m_PreviousMacLicense[33];
WCHAR m_SoftwareName[16];
WCHAR m_ClientName[65];
BYTE m_Version;
BYTE m_EncryptionLength;
time_t m_LicenseTime;
};
I have tried several methods of replicating this in C#, firstly reading the file member by member, for example:
BinaryReader reader = new BinaryReader(licenceFileStream, Encoding.BigEndianUnicode);
licenceFileData.EncryptedKey = reader.ReadBytes(licenceFileData.EncryptedKey.Length);
byte[] mac = new byte[sizeof(char) * licenceFileData.MacAddress.Length];
mac = reader.ReadBytes(mac.Length);
Something strange I noticed about this method is that a lot of the values were being set as 204 seemingly arbitrarily, when they are read correctly in the C++ application, so I've had to add this:
if (mac[0] == 204)
mac[0] = 0;
Which makes no sense to me, it's not an alignment issue, reading from a byte before or after that gives totally garbage values (chinese characters) and this check and assigning 0 to the first index allows me to get the proper value.
Another method I've tried was found on here on StackOverflow, and is as follows:
public static T ReadStruct<T>(this BinaryReader reader) where T : struct
{
Byte[] buffer = new Byte[Marshal.SizeOf(typeof(T))];
reader.Read(buffer, 0, buffer.Length);
GCHandle handle = default(GCHandle);
try
{
handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
return (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
}
finally
{
if (handle.IsAllocated)
handle.Free();
}
}
Oddly, this method also returns some values as 204 where the C++ application reads them as the correct value.
The struct I'm using in C# looks like this:
unsafe struct LicenceFileDataS
{
internal fixed byte m_EncryptedKey[255];
internal fixed char m_MacAddress[33];
internal fixed char m_PreviousMacLicense[33];
internal fixed char m_SoftwareName[16];
internal fixed char m_ClientName[65];
internal byte m_Version;
internal byte m_EncryptionLength;
internal long m_LicenseTime;
}
Does anyone know a better method for porting this code or how I can fix the values that are being read as 204? It works perfectly in C++. My apologies for the wall of text.

How do I call this c function in c# (unmarshalling return struct)?

I want to use c# interop to call a function from a dll written in c. I have the header files.
Take a look at this:
enum CTMBeginTransactionError {
CTM_BEGIN_TRX_SUCCESS = 0,
CTM_BEGIN_TRX_ERROR_ALREADY_IN_PROGRESS,
CTM_BEGIN_TRX_ERROR_NOT_CONNECTED
};
#pragma pack(push)
#pragma pack(1)
struct CTMBeginTransactionResult {
char * szTransactionID;
enum CTMBeginTransactionError error;
};
struct CTMBeginTransactionResult ctm_begin_customer_transaction(const char * szTransactionID);
How do I call ctm_begin_customer_transaction from c#. The const char * mapps well to string, but despite various attempts (looking at stackoverflow and other sites), I fail to marshal the return structure. If I define the function to return IntPtr it works ok...
Edit
I changed the return type to IntPtr and use:
CTMBeginTransactionResult structure = (CTMBeginTransactionResult)Marshal.PtrToStructure(ptr, typeof(CTMBeginTransactionResult));
but it throws AccessViolationException
I also tried:
IntPtr ptr = Transactions.ctm_begin_customer_transaction("");
int size = 50;
byte[] byteArray = new byte[size];
Marshal.Copy(ptr, byteArray, 0, size);
string stringData = Encoding.ASCII.GetString(byteArray);
stringData == "70e3589b-2de0-4d1e-978d-55e22225be95\0\"\0\0\a\0\0\b\b?" at this point. The "70e3589b-2de0-4d1e-978d-55e22225be95" is the szTransactionID from the struct. Where is the Enum? Is it the next byte?

There's a memory management problem hidden in this struct. Who owns the C string pointer? The pinvoke marshaller will always assume that the caller owns it so it will try to release the string. And passes the pointer to CoTaskMemFree(), same function as the one called by Marshal.FreeCoTaskMem(). These functions use the COM memory allocator, the universal interop memory manager in Windows.
This rarely comes to a good end, C code does not typically use that allocator unless the programmer designed his code with interop in mind. In which case he'd never have used a struct as a return value, interop always works much less trouble-free when the caller supplies buffers.
So you cannot afford to let the marshaller do its normal duty. You must declare the return value type as IntPtr so it doesn't try to release the string. And you must marshal it yourself with Marshal.PtrToStructure().
That however still leaves the question unanswered, who owns the string? There is nothing you can do to release the string buffer, you don't have access to the allocator used in the C code. The only hope you have is that the string wasn't actually allocated on the heap. That's possible, the C program might be using string literals. You need to verify that guess. Call the function a billion times in a test program. If that doesn't explode the program then you're good. If not then only C++/CLI can solve your problem. Given the nature of the string, a "transaction ID" ought to change a lot, I'd say you do have a problem.

I hate to answer my own question, but I found the solution to marshal the resulting struct. The struct is 8 bytes long (4 bytes for the char * and 4 bytes for enum). Marshalling the string does not work automatically, but the following works:
// Native (unmanaged)
public enum CTMBeginTransactionError
{
CTM_BEGIN_TRX_SUCCESS = 0,
CTM_BEGIN_TRX_ERROR_ALREADY_IN_PROGRESS,
CTM_BEGIN_TRX_ERROR_NOT_CONNECTED
};
// Native (unmanaged)
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
internal struct CTMBeginTransactionResult
{
public IntPtr szTransactionID;
public CTMBeginTransactionError error;
};
// Managed wrapper around native struct
public class BeginTransactionResult
{
public string TransactionID;
public CTMBeginTransactionError Error;
internal BeginTransactionResult(CTMBeginTransactionResult nativeStruct)
{
// Manually marshal the string
if (nativeStruct.szTransactionID == IntPtr.Zero) this.TransactionID = "";
else this.TransactionID = Marshal.PtrToStringAnsi(nativeStruct.szTransactionID);
this.Error = nativeStruct.error;
}
}
[DllImport("libctmclient-0.dll")]
internal static extern CTMBeginTransactionResult ctm_begin_customer_transaction(string ptr);
public static BeginTransactionResult BeginCustomerTransaction(string transactionId)
{
CTMBeginTransactionResult nativeResult = Transactions.ctm_begin_customer_transaction(transactionId);
return new BeginTransactionResult(nativeResult);
}
The code works, but I still need to investigate, if calling the unmanaged code results in memory leaks.

C# calling C function that returns struct with fixed size char array

So, there have been many variants of this question, and after looking at several I still can't figure it out.
This is the C code:
typedef struct
{
unsigned long Identifier;
char Name[128];
} Frame;
Frame GetFrame(int index);
This is the C# code:
struct Frame
{
public ulong Identifier;
[MarshalAs(UnmanagedType.ByValArray, ArraySubType = UnmanagedType.I1, SizeConst = 128)]
public char[] Name;
}
[DllImport("XNETDB.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern Frame GetFrame(int index);
This is the last attempt I tried in C#, and it seems pretty logical, but I get the error "Method's signature is not PInvoke compatible." So, I'm kind of lost on what to try next. Any help is appreciated.
Thanks,
Kevin
Updated Kevin added this as an edit to my answer
I should instead change my C code:
void GetFrame(int index, Frame * f);
and use instead for C#:
struct Frame
{
public uint Identifier;
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 128)]
public string Name;
}
[DllImport("XNETDB.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern void GetFrame(int index, ref Frame f);

There are two problems with the PInvoke signature that you've chosen.
The first is easy to fix. You have a mistranslation of unsigned long. In C an unsigned long is typically only 4 bytes. You chose the C# type long which is 8 bytes. Changing the C# code to use uint will fix this.
The second is a bit harder. As Tergiver pointed out the CLR Marshaller only supports a struct in the return position if it's blittable. Blittable is a fancy way of saying that it has the exact same memory representation in native and managed code. The struct definition you've chosen isn't blittable because it has a nested array.
This can be worked around though if you remember that PInvoke is a very simple process. The CLR Marshaller really just needs you to answer 2 questions with the signature of your types and pinvoke methods
How many bytes am I copying?
In which direction do they need to go?
In this case the number of bytes is sizeof(unsigned long) + 128 == 132. So all we need to do is build up a managed type that is blittable and has a size of 132 bytes. The easiest way to do this is to define a blob to handle the array portion
[StructLayout(LayoutKind.Sequential, Size = 128)]
struct Blob
{
// Intentionally left empty. It's just a blob
}
This is a struct with no members that will appear to the marshaller as having a size of 128 bytes (and as a bonus it's blittable!). Now we can easily define the Frame structure as a combination of an uint and this type
struct Frame
{
public int Identifier;
public Blob NameBlob;
...
}
Now we have a blittable type with a size the marshaller will see as 132 bytes. This means it will work just fine with the GetFrame signature you've defined
The only part left is giving you access to the actual char[] for the name. This is a bit tricky but can be solved with a bit of marshal magic.
public string GetName()
{
IntPtr ptr = IntPtr.Zero;
try
{
ptr = Marshal.AllocHGlobal(128);
Marshal.StructureToPtr(NameBlob, ptr, false);
return Marshal.PtrToStringAnsi(ptr, 128);
}
finally
{
if (ptr != IntPtr.Zero)
{
Marshal.FreeHGlobal(ptr);
}
}
}
Note: I can't comment on the calling convention portion because I'm unfamiliar with the GetFrame API but that's something i would definitely check on.

The problem is that the native function returns a non-blittable type as a return value.
http://msdn.microsoft.com/en-us/library/ef4c3t39.aspx
P/Invoke cannot have non-blittable types as a return value.
You cannot p/Invoke that method. [EDIT It is actually possible, see JaredPar's answer]
Returning 132 bytes by value is a bad idea. If this native code is yours, I would fix it. You can fix it by allocating the 132 bytes and returning a pointer. Then add a FreeFrame method to release that memory. Now it can be p/Invoked.
Alternately, you could change it to accept a pointer to the Frame memory that it will fill in.

Another option to JaredPar's is to utilize the C# fixed size buffer feature. This does however require you to turn on the setting to allow unsafe code, but it avoids having 2 structs.
class Program
{
private const int SIZE = 128;
unsafe public struct Frame
{
public uint Identifier;
public fixed byte Name[SIZE];
}
[DllImport("PinvokeTest2.DLL", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
private static extern Frame GetFrame(int index);
static unsafe string GetNameFromFrame(Frame frame)
{
//Option 1: Use if the string in the buffer is always null terminated
//return Marshal.PtrToStringAnsi(new IntPtr(frame.Name));
//Option 2: Use if the string might not be null terminated for any reason,
//like if were 128 non-null characters, or the buffer has corrupt data.
return Marshal.PtrToStringAnsi(new IntPtr(frame.Name), SIZE).Split('\0')[0];
}
static void Main()
{
Frame a = GetFrame(0);
Console.WriteLine(GetNameFromFrame(a));
}
}

Marshalling Issues

I have a C++ DLL that interacts with a card reader. It requires a pointer to a data struct, which isn't a problem to do. However, when trying to interact with the DLL in C# I'm getting all kinds of problems. Errors writing to protected memory, the application just shutting down after executing the getData command, etc. Here's what we have.
C++ Method from header
void readCard(cardData* dataBuffer);
C# code
Wrapper.cs
public struct cardData{
Byte[] data01;
Byte[] data02;
}
[dllImport("card.dll")]
public static extern void readCard(ref cardData data);
form1.cs
Wrapper.cardData tmpData = new wrapper.cardData();
tmpData.data01 = new Byte[];
tmpData.data02 = new Byte[];
readCard(ref tmpData);
I've also tried passing cardData as an IntPtr using Marshal.StructureToPtr, which didn't return any data when I returned tried to read the ptr into a struct Marshal.PtrToStructure...
I've been trying to work this out using the help files and other posts because it seems that alot of people have trouble trying to work with C/C++ DLLs. I even tried to write the whole thing in C++ and have it return a string with the data parsed in the C++ DLL but that throws a reading/writing to protected memory error

The biggest problem I see with your code is that you have not given your byte[] members an explicit size. Without this size operator, the marshaller will treat them just like a simple reference type. The resulting struct will have a size of 8 bytes on a 32 bit platform and will almost certainly lead to writing of protected memory.
Assuming the byte arrays are of a fixed size in the C code, you should use the MarshalAs attribute to give the byte arrays the same semantics in managed code. This does entail giving them a fixed size.
[StructLayout(LayoutKind.Sequential)]
public struct cardData{
[MarshalAs(UnmanagedType.ByValArray, SizeConst=300)]
Byte[] data01;
[MarshalAs(UnmanagedType.ByValArray, SizeConst=300)]
Byte[] data02;
}
Change 300 to be whatever size is specified in the native code for the array.
Also you should add the StructLayout attribute as well.

Maybe your cardData needs to use the StructLayoutAttribute. Also you can use Dependency Walker to find the location in card.dll for that method, and add that as a named parameter.

I noticed that your Byte[] have no size associated with them. Do you know what size the arrays are supposed to be? IIRC, the space for the arrays needs to allocated when they're initialized.
If I'm way off base, let me know and I'll delete this post.

The PInvoke Signature Toolkit has helped me in the past.
For example the following C/C++:
struct cardData{
byte[] data01;
byte[] data02;
}
void readCard(cardData* dataBuffer);
It has:
System.Runtime.InteropServices.StructLayoutAttribute( System.Runtime.InteropServices.LayoutKind.Sequential)]
public struct cardData {
/// byte[]
public byte[] data01;
/// byte[]
public byte[] data02;
}
/// Return Type: cardData
///dataBuffer: cardData*
public delegate cardData readCard(ref cardData dataBuffer);

Here's a snippet of a C-DLL wrapper in C# that I've made.
Like Yuriy mentioned, you're missing a StructLayout attribute, and you probably ought to be using native types in your struct and your function declaration. This will probably require you to use the unsafe keyword in a few places, which may or may not be acceptable to you - but it was fine for me.
[StructLayout(LayoutKind.Sequential)]
public unsafe struct X_Message
{
public byte id;
public byte* data;
public DWORD data_length;
}
// ...
[DllImport("x-driver.dll")]
protected unsafe static extern int X_ReadMessage(void* h, X_Message* message);

Ok. So the struct has been set as suggested with the struct layout.
Wrapper.cs
[StructLayout(LayoutKind.Sequential)]
public struct cardData{
[MarshalAs(UnmanagedType.ByValArray, SizeConst=99)]
Byte[] data01;
[MarshalAs(UnmanagedType.ByValArray, SizeConst=101)]
Byte[] data02;
}
[DllImport("card.Dll")]
public static extern void readCard(ref cardData data);
And now it just closes... No errors, no change in data, the app just shuts down.

Use IntPtr instead of byte[]. Your DLL can't handle managed data.

Call unmanaged Code from C# - returning a struct with arrays

[EDIT] I changed the source as suggested by Stephen Martin (highlighted in bold). And added the C++ source code as well.
I'd like to call an unmanaged function in a self-written C++ dll. This library reads the machine's shared memory for status information of a third party software. Since there are a couple of values, I'd like to return the values in a struct. However, within the struct there are char [] (Arrays of char with a fixed size). I now try to receive that struct from the dll call like this:
[StructLayout(LayoutKind.Sequential)]
public struct SYSTEM_OUTPUT
{
UInt16 ReadyForConnect;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 128)]
String VersionStr;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 1024)]
String NameOfFile;
// actually more of those
}
public partial class Form1 : Form
{
public SYSTEM_OUTPUT output;
[DllImport("testeshm.dll", EntryPoint="getStatus")]
public extern static int getStatus(out SYSTEM_OUTPUT output);
public Form1()
{
InitializeComponent();
}
private void ReadSharedMem_Click(object sender, EventArgs e)
{
try
{
label1.Text = getStatus(out output).ToString();
}
catch (AccessViolationException ave)
{
label1.Text = ave.Message;
}
}
}
I will post code from the c++ dll as well, I'm sure there's more to hunt down. The original struct STATUS_DATA has an array of four instances of the struct SYSTEM_CHARACTERISTICS and within that struct there are char[]s, that are not being filled (yet), resulting in a bad pointer. That's why I'm trying to extract a subset of the first SYSTEM_CHARACTERISTICS item in STATUS_DATA.
#include <windows.h>
#include <stdio.h>
#include <conio.h>
#include <tchar.h>
#include <iostream>
#if defined(_MSC_VER)
#include <windows.h>
#define DLL extern "C" __declspec(dllexport)
#else
#define DLL
#endif
using namespace std;
enum { SYSID_LEN = 1024, VERS_LEN = 128, SCENE_LEN = 1024 };
enum { MAX_ENGINES = 4 };
struct SYSTEM_CHARACTERISTICS
{
unsigned short ReadyForConnect;
char VizVersionStr[VERS_LEN];
char NameOfFile[SCENE_LEN];
char Unimplemented[SCENE_LEN]; // not implemented yet, resulting to bad pointer, which I want to exclude (reason to have SYSTEM_OUTPUT)
};
struct SYSTEM_OUTPUT
{
unsigned short ReadyForConnect;
char VizVersionStr[VERS_LEN];
char NameOfFile[SCENE_LEN];
};
struct STATUS_DATA
{
SYSTEM_CHARACTERISTICS engine[MAX_ENGINES];
};
TCHAR szName[]=TEXT("E_STATUS");
DLL int getStatus(SYSTEM_OUTPUT* output)
{
HANDLE hMapFile;
STATUS_DATA* pBuf;
hMapFile = OpenFileMapping(
FILE_MAP_READ, // read access
FALSE, // do not inherit the name
szName); // name of mapping object
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not open file mapping object (%d).\n"),
GetLastError());
return -2;
}
pBuf = (STATUS_DATA*) MapViewOfFile(hMapFile, FILE_MAP_READ, 0, 0, 0);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"),
GetLastError());
CloseHandle(hMapFile);
return -1;
}
output->ReadyForConnect = pBuf->engine[0].ReadyForConnect;
memcpy(output->VizVersionStr, pBuf->engine[0].VizVersionStr, sizeof(pBuf->engine[0].VizVersionStr));
memcpy(output->NameOfFile, pBuf->engine[0].NameOfFile, sizeof(pBuf->engine[0].NameOfFile));
CloseHandle(hMapFile);
UnmapViewOfFile(pBuf);
return 0;
}
Now I'm getting an empty output struct and the return value ist not 0 as intended. It is rather a changing number with seven digits, which leaves me puzzled... Have I messed up in the dll? If I make the unmanaged code executable and debug it, I can see, that output is being filled with the appropriate values.

Make sure your ReadyForConnect field is not filled up to 4 bytes. In my project it turned out all short int (2 bytes) fields were filled with dummy bytes to 4 bytes in unmanaged DLL. If that's the issue, you should marshall the struct this way:
[StructLayout(LayoutKind.Sequential)]
public struct SYSTEM_OUTPUT
{
[MarshalAs(UnmanagedType.I2)]
UInt16 ReadyForConnect;
[MarshalAs(UnmanagedType.ByValArray, ArraySubType=UnmanagedType.I1, SizeConst=2)]
byte[] aligment; // 2 byte aligment up to 4 bytes margin
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 128)]
String VersionStr;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 1024)]
String NameOfFile; // ...
}
If the strings are ANSI null terminated strings you can annotate them as:
[MarshalAs(UnmanagedType.LPStr)] public String VersionStr;

When returning information in a struct the standard method is to pass a pointer to a struct as a parameter of the method. The method fills in the struct members and then returns a status code (or boolean) of some kind. So you probably want to change your C++ method to take a SYSTEM_OUTPUT* and return 0 for success or some error code:
public partial class Form1 : Form
{
public SYSTEM_OUTPUT output;
[DllImport("testeshm.dll", EntryPoint="getStatus")]
public extern static int getStatus(out SYSTEM_OUTPUT output);
public Form1()
{
InitializeComponent();
}
private void ReadSharedMem_Click(object sender, EventArgs e)
{
try
{
if(getStatus(out output) != 0)
{
//Do something about error.
}
}
catch (AccessViolationException ave)
{
label1.Text = ave.Message;
}
}
}

You aren't actually marshaling any data over to the managed side. When you declare output on the managed side, it's default value is null. Then, on the unmanaged side, you never allocate any memory for output. You should allocate some unmanaged memory, pass the pointer to that memory to your dll function, then marshal the pointer for that memory to your struct:
[StructLayout(LayoutKind.Sequential, Pack = 4, CharSet = CharSet.Ansi)]
public struct SYSTEM_OUTPUT
{
UInt16 ReadyForConnect;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 128)]
String VersionStr;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 1024)]
String NameOfFile;
// actually more of those
}
public partial class Form1 : Form
{
public SYSTEM_OUTPUT output;
[DllImport("testeshm.dll", EntryPoint="getStatus")]
public extern static int getStatus(IntPtr output);
public Form1()
{
InitializeComponent();
}
private void ReadSharedMem_Click(object sender, EventArgs e)
{
IntPtr ptr;
try
{
ptr = Marshall.AllocHGlobal(Marshall.SizeOf(typeof(SYSTEM_OUTPUT)));
int ret = getStatus(ptr);
if(ret == 0)
{
output = (SYSTEM_OUTPUT)Marshal.PtrToStructure(ptr, typeof(SYSTEM_OUTPUT));
}
//do something with output
label1.Text = ret;
}
catch (AccessViolationException ave)
{
label1.Text = ave.Message;
}
finally
{
Marshal.FreeHGlobal(ptr); //make sure to free the memory
}
}
}
Edit:
Your problem could be an issue with the difference between packing strategies. I've updated the struct definition.

EDIT: I am rewriting this whole answer.
I took all of both your C++ and C# code, dropped it into a solution and ran it -- and everything works for me. I didn't have your specific memory mapping stuff so I simulated it by filling pBuf with some fake data, and everything makes it back fine; both the return value and the output struct are correct.
Could something be amiss with your project settings? This sounds silly, but you mentioned running and debugging the unamnaged code; you are building a dll right?

What are you trying to do is possible, but I think you are solving the wrong problem.
Why not read the memory mapped file direct from C#?
Take a look at Winterdom.IO.FileMap
I have used it and it works fine.
MemoryMappedFile file = MemoryMappedFile.Open(FileMapRead, name);
using (Stream stream = memoryMappedFile.MapView(MapAccess.FileMapAllAccess, 0, length))
{
// here read the information that you need
}
With that you are not finished - you still have to convert a byte buffer to a struct, but you are all on the managed side and it will be easier.

who allocated the memory for the structure? You cannot delete native memory from the managed heap. Generally speaking the native DLL should allocate on the COM heap if it expect the caller to free the memory, or return a callback interface like IMalloc to free the returning memory. That means you need to receive the result memory address as IntPtr and use System.Runtime.InteropServices.Marshal to copy data from native to managed heap (may be to a structure) before freeing the memory.
Edit for the updated function signature:
use public static extern int getStatus(ref SYSTEM_OUTPUT output); You are not allocating on the COM heap in the native function, so out is unnecessary.

Have you considered adding a C++/CLI assembly to your project? That's an extremely easy and powerful way to bridge the gap between managed and unmanaged code. I use it quite a lot myself.