Develop Reference

Marshalling C++ Struct with function pointers in C#

Im making a C# managed extension dll for NPS witch looks like this:
public enum RADIUS_CODE
{
rcUnknown = 0,
rcAccessRequest = 1,
rcAccessAccept = 2,
rcAccessReject = 3,
rcAccountingRequest = 4,
rcAccountingResponse = 5,
rcAccessChallenge = 11,
rcDiscard = 256
}
public enum RADIUS_EXTENSION_POINT
{
repAuthentication,
repAuthorization
}
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
public delegate ref RADIUS_ATTRIBUTE_ARRAY GetRequestDelegate(ref RADIUS_EXTENSION_CONTROL_BLOCK ECB);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
public delegate ref RADIUS_ATTRIBUTE_ARRAY GetResponseDelegate(ref RADIUS_EXTENSION_CONTROL_BLOCK ECB, RADIUS_CODE rcResponseType);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
public delegate uint SetResponseTypeDelegate(ref RADIUS_EXTENSION_CONTROL_BLOCK ECB, RADIUS_CODE rcResponseType);
[StructLayout(LayoutKind.Sequential)]
public struct RADIUS_EXTENSION_CONTROL_BLOCK
{
public uint cbSize;
public uint dwVersion;
public RADIUS_EXTENSION_POINT repPoint;
public RADIUS_CODE rcRequestType;
public RADIUS_CODE rcResponseType;
public GetRequestDelegate GetRequest;
public GetResponseDelegate GetResponse;
public SetResponseTypeDelegate SetResponseType;
}
[DllExport("RadiusExtensionProcess2", CallingConvention = CallingConvention.Winapi)]
public static uint RadiusExtensionProcess2(ref RADIUS_EXTENSION_CONTROL_BLOCK pECB)
{
//var result = pECB.SetResponseType(ref pECB, RADIUS_CODE.rcAccessAccept);
return 0;
}
NPS passed a control block to the function that contains 5 fields and 3 pointers to functions, the fields map correctly but i cant get the functions working.
If I uncomment the line before the return NPS discard the request noting that the dll crashed, with the line commented out NPS processes the request successfully.
I'm well versed in assembly and C# but a newbie at C++ and managed/unmamaged interop.
I've tried all sorts of marshaling methods, calling conventions that i could find and stepping thru the disassembly but this is as far as I've gotten after a week and I'm out of ideas...
The documentation I've been going by can be found here
Some Progress
I had previously tried Marshal.GetDelegateForFunctionPointer as suggested by #RonBeyer with no success but it's working now, i think i was missing some
ref keywords at that time, so this seem to be working
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
public delegate ref RADIUS_ATTRIBUTE AttributeAtDelegate([In] ref RADIUS_ATTRIBUTE_ARRAY Array, [In] uint dwIndex);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
public delegate uint GetSizeDelegate([In] ref RADIUS_ATTRIBUTE_ARRAY Array);
[StructLayout(LayoutKind.Sequential)]
public struct RADIUS_ATTRIBUTE_ARRAY
{
public uint cbSize;
public IntPtr Add;
public IntPtr AttributeAt;
public IntPtr GetSize;
public IntPtr InsertAt;
public IntPtr RemoveAt;
public IntPtr SetAt;
}
[StructLayout(LayoutKind.Sequential)]
public struct RADIUS_ATTRIBUTE
{
public uint dwAttrType;
public RADIUS_DATA_TYPE fDataType;
public uint cbDataLength;
public IntPtr lpValue;
}
[StructLayout(LayoutKind.Sequential)]
public struct RADIUS_EXTENSION_CONTROL_BLOCK
{
public uint cbSize;
public uint dwVersion;
public RADIUS_EXTENSION_POINT repPoint;
public RADIUS_CODE rcRequestType;
public RADIUS_CODE rcResponseType;
public IntPtr GetRequest;
public IntPtr GetResponse;
public IntPtr SetResponseType;
}
[DllExport("RadiusExtensionProcess2", CallingConvention = CallingConvention.Winapi)]
public static uint RadiusExtensionProcess2(ref RADIUS_EXTENSION_CONTROL_BLOCK pECB)
{
GetRequestDelegate GetRequest = (GetRequestDelegate)Marshal.GetDelegateForFunctionPointer(pECB.GetRequest, typeof(GetRequestDelegate));
GetResponseDelegate GetResponse = (GetResponseDelegate)Marshal.GetDelegateForFunctionPointer(pECB.GetResponse, typeof(GetResponseDelegate));
SetResponseTypeDelegate SetResponseType = (SetResponseTypeDelegate)Marshal.GetDelegateForFunctionPointer(pECB.SetResponseType, typeof(SetResponseTypeDelegate));
uint SetResponseTypeResult = SetResponseType(ref pECB, RADIUS_CODE.rcAccessAccept); //SetResponseTypeResult is 0 and NPS sends a access granted message
RADIUS_ATTRIBUTE_ARRAY AttrArray = GetRequest(ref pECB);
return 0;
}
SetResponseType works without crashing the dll and GetRequest returns the expected value for cbSize and what look like valid pointers for the functions defined in the RADIUS_ATTRIBUTE_ARRAY struct.
So i proceeded to map AttributeAt and GetSize the same way but the data they return is nonsence.
I double and triple checked the definitions here and here but so far i haven't been able to get the proper data from these functions, since the dll dosen't crash and i dont get null back, i can only assume there is something wrong with how I'm passing the parameters to the functions
On that note, are the in/out keywords in the C++ function definitions just info for developers or do they effect how parameters are handled on the stack?

Getting Symbols from debugged process MainModule

I started writing a debugger in C#, to debug any process on my operating system. For now, it only can handle breakpoints (HW, SW, and Memory), but now I wanted to show the opcode of the process.
My first attempt was with nidsasm (NASM), but this is not suitable, because after startup a.Net Application assembler instructions are different from ndisasm (tested with CheatEngine).
So I searched a while and found some methods from the dbghelp.dll which can be called to list all loaded modules and symbols (plus the base address). Ok, my attempt is, to disassemble all modules separately with SharpDisasm.
I use ProcessModuleCollection modules = ProcessData.Instance.MPMR.ReadProcess.Modules; to get all loaded modules of the process. This works perfectly.
Now I tried to load the symbols of the MainModule, but at this point, I stuck with the implementation. I implemented the SymEnumSymbols Function with p/Invoke and other necessary functions like SymInitialize.
When I call it with a BaseAddress of for example the "User32.dll", all symbols are printed perfectly, but for the MainModule, I didn't get any symbols.
This is a screenshot from CheatEngine:
Symbols gained from Cheat Engine
As you can see, there are symbols like "Form1_Load", which I don't get with my implementation.
This is the necessary code sample:
if (!DebugApi.SymInitialize(ProcessData.Instance.MPMR.M_hProcess, null, false))
{
var err = Marshal.GetLastWin32Error();
//throw new Exception("GetMemoryInfo failed : GetLastError() : " + new Win32Exception(err).Message);
Console.WriteLine("GetMemoryInfo failed : GetLastError() : " + new Win32Exception(err).Message);
return;
}
if (!DebugApi.SymEnumSymbols(ProcessData.Instance.MPMR.M_hProcess, (ulong)ProcessData.Instance.MPMR.ReadProcess.MainModule.BaseAddress, "!", DebugApi.EnumSyms, IntPtr.Zero))
{
var err = Marshal.GetLastWin32Error();
//throw new Exception("GetMemoryInfo failed : GetLastError() : " + new Win32Exception(err).Message);
Console.WriteLine("GetMemoryInfo failed : GetLastError() : " + new Win32Exception(err).Message);
return;
}
DebugApi.SymCleanup(ProcessData.Instance.MPMR.M_hProcess);
And my DebugApi, with all necessary p/Invoke functions.
public class DebugApi
{
[DllImport("dbghelp.dll", SetLastError = true, CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool SymInitialize(IntPtr hProcess, string UserSearchPath, [MarshalAs(UnmanagedType.Bool)]bool fInvadeProcess);
[DllImport("dbghelp.dll", SetLastError = true, CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool SymCleanup(IntPtr hProcess);
[DllImport("dbghelp.dll", SetLastError = true, CharSet = CharSet.Unicode)]
public static extern ulong SymLoadModuleEx(IntPtr hProcess, IntPtr hFile, string ImageName, string ModuleName, long BaseOfDll, int DllSize, IntPtr Data, int Flags);
[DllImport("dbghelp.dll", SetLastError = true, CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool SymEnumSymbols(IntPtr hProcess, ulong BaseOfDll, string Mask, PSYM_ENUMERATESYMBOLS_CALLBACK EnumSymbolsCallback, IntPtr UserContext);
public delegate bool PSYM_ENUMERATESYMBOLS_CALLBACK(ref SYMBOL_INFO pSymInfo, uint SymbolSize, IntPtr UserContext);
public static bool EnumSyms(ref SYMBOL_INFO pSymInfo, uint SymbolSize, IntPtr UserContext)
{
Console.Out.WriteLine("Name: " + pSymInfo.Name);
return true;
}
[Flags]
public enum SymFlag : uint
{
VALUEPRESENT = 0x00000001,
REGISTER = 0x00000008,
REGREL = 0x00000010,
FRAMEREL = 0x00000020,
PARAMETER = 0x00000040,
LOCAL = 0x00000080,
CONSTANT = 0x00000100,
EXPORT = 0x00000200,
FORWARDER = 0x00000400,
FUNCTION = 0x00000800,
VIRTUAL = 0x00001000,
THUNK = 0x00002000,
TLSREL = 0x00004000,
}
[Flags]
public enum SymTagEnum : uint
{
Null,
Exe,
Compiland,
CompilandDetails,
CompilandEnv,
Function,
Block,
Data,
Annotation,
Label,
PublicSymbol,
UDT,
Enum,
FunctionType,
PointerType,
ArrayType,
BaseType,
Typedef,
BaseClass,
Friend,
FunctionArgType,
FuncDebugStart,
FuncDebugEnd,
UsingNamespace,
VTableShape,
VTable,
Custom,
Thunk,
CustomType,
ManagedType,
Dimension
};
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
public struct SYMBOL_INFO
{
public uint SizeOfStruct;
public uint TypeIndex;
public ulong Reserved1;
public ulong Reserved2;
public uint Reserved3;
public uint Size;
public ulong ModBase;
public SymFlag Flags;
public ulong Value;
public ulong Address;
public uint Register;
public uint Scope;
public SymTagEnum Tag;
public int NameLen;
public int MaxNameLen;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 1024)]
public string Name;
}
}
My Functions should be ok, because it works with other Modules (loaded dll's for example). Maybe I don't understand the concept of symbols of a .Net Executable or missing something.

can it be that you are looking for System.Diagnostics.SymbolStore.ISymbolScope.
Have a look at the class SymbolAccess, you can use it to gain access to ISymbolScope.GetLocals() that returns ISymbolVariable[] and GetChildren() again returning in this time an array called ISymbolVariable[]
Now another interesting set of reference code samples is the Debugger extension lets you "snif" the values as shown here

Automatically a debug process from C# code and read register values

I'm looking for a way to read the edx registry at a certain address like asked in this question: Read eax register
Although my solution needs to be in C# and I tried to make it, what I got at this moment is:
public static IntPtr GetEdx(IntPtr address, Process process)
{
const uint DBG_EXCEPTION_NOT_HANDLED = 0x80010001;
const uint EXCEPTION_SINGLE_STEP = 0x80000004;
const int DBG_CONTINUE = 0x00010002; // Seems to work better than DBG_EXCEPTION_NOT_HANDLED
//DebugSetProcessKillOnExit(0);
DEBUG_EVENT evt = new DEBUG_EVENT();
// Attach to the process we provided the thread as an argument
if (!DebugActiveProcess(process.Id))
throw new Win32Exception();
CONTEXT context = new CONTEXT();
foreach (ProcessThread thread in process.Threads)
{
uint iThreadId = (uint)thread.Id;
IntPtr hThread =
OpenThread(
ThreadAccessFlags.SUSPEND_RESUME | ThreadAccessFlags.SET_CONTEXT |
ThreadAccessFlags.GET_CONTEXT, false, iThreadId);
// Suspent the thread
if (SuspendThread(hThread) == -1) throw new ApplicationException("Cannot suspend thread.");
context = new CONTEXT
{
ContextFlags = (uint)CONTEXT_FLAGS.CONTEXT_DEBUG_REGISTERS |
(uint)CONTEXT_FLAGS.CONTEXT_INTEGER
};
// Get the context
if (!GetThreadContext(hThread, ref context))
throw new Win32Exception();
// Change the context
context.Dr0 = (uint)address;
context.Dr7 = 0x00000001;
// Set the changed context back
if (!SetThreadContext(hThread, ref context))
throw new Win32Exception();
// Check if setting the context give any errors
var error = Marshal.GetLastWin32Error();
if (error != 0)
{
throw new ApplicationException("Error is setting context.");
}
// Resume the thread
if (ResumeThread(hThread) == -1) throw new ApplicationException("Cannot resume thread.");
}
while (true)
{
if (!WaitForDebugEvent(out evt, -1))
throw new Win32Exception();
// Multiple if's for easier debugging at this moment
if (evt.dwDebugEventCode == (uint)DebugEventType.EXCEPTION_DEBUG_EVENT)
{
if (evt.Exception.ExceptionRecord.ExceptionCode == EXCEPTION_SINGLE_STEP)
{
if (evt.Exception.ExceptionRecord.ExceptionAddress == address)
{
context = new CONTEXT
{
ContextFlags = (uint)CONTEXT_FLAGS.CONTEXT_DEBUG_REGISTERS |
(uint)CONTEXT_FLAGS.CONTEXT_INTEGER
};
GetThreadContext((IntPtr)evt.dwThreadId, ref context);
return (IntPtr)context.Ebx; // ebx get
}
}
}
ContinueDebugEvent(evt.dwProcessId, evt.dwThreadId, DBG_CONTINUE);//DBG_EXCEPTION_NOT_HANDLED);
}
}
With a whole lot of Kernel32 methods:
[DllImport("kernel32.dll")]
static extern int ResumeThread(IntPtr hThread);
[DllImport("kernel32.dll")]
static extern uint SuspendThread(IntPtr hThread);
[DllImport("kernel32.dll")]
public static extern IntPtr OpenThread(ThreadAccessFlags dwDesiredAccess, bool bInheritHandle, uint dwThreadId);
[DllImport("Kernel32.dll", SetLastError = true)]
static extern bool DebugActiveProcess(int dwProcessId);
[DllImport("Kernel32.dll", SetLastError = true)]
static extern bool WaitForDebugEvent([Out] out DEBUG_EVENT lpDebugEvent, int dwMilliseconds);
[DllImport("Kernel32.dll", SetLastError = true)]
static extern bool ContinueDebugEvent(int dwProcessId, int dwThreadId, uint dwContinueStatus);
[DllImport("Kernel32.dll", SetLastError = true)]
public static extern bool IsDebuggerPresent();
[DllImport("kernel32.dll")]
private static extern bool GetThreadContext(IntPtr hThread, ref CONTEXT lpContext);
[DllImport("kernel32.dll")]
public static extern bool SetThreadContext(IntPtr hThread, ref CONTEXT lpContext);
[StructLayout(LayoutKind.Sequential)]
public unsafe struct DEBUG_EVENT
{
public readonly uint dwDebugEventCode;
public readonly int dwProcessId;
public readonly int dwThreadId;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 86, ArraySubType = UnmanagedType.U1)]
private readonly byte[] debugInfo;
public EXCEPTION_DEBUG_INFO Exception
{
get
{
if (debugInfo == null)
return new EXCEPTION_DEBUG_INFO();
fixed (byte* ptr = debugInfo)
{
return *(EXCEPTION_DEBUG_INFO*)ptr;
}
}
}
public LOAD_DLL_DEBUG_INFO LoadDll
{
get
{
if (debugInfo == null)
return new LOAD_DLL_DEBUG_INFO();
fixed (byte* ptr = debugInfo)
{
return *(LOAD_DLL_DEBUG_INFO*)ptr;
}
}
}
}
[StructLayout(LayoutKind.Sequential)]
public struct LOAD_DLL_DEBUG_INFO
{
public readonly IntPtr hFile;
public readonly IntPtr lpBaseOfDll;
public readonly uint dwDebugInfoFileOffset;
public readonly uint nDebugInfoSize;
public readonly IntPtr lpImageName;
public readonly ushort fUnicode;
}
[StructLayout(LayoutKind.Sequential)]
public struct EXCEPTION_DEBUG_INFO
{
public EXCEPTION_RECORD ExceptionRecord;
public readonly uint dwFirstChance;
}
[StructLayout(LayoutKind.Sequential)]
public struct EXCEPTION_RECORD
{
public readonly uint ExceptionCode;
public readonly uint ExceptionFlags;
public readonly IntPtr ExceptionRecord;
public readonly IntPtr ExceptionAddress;
public readonly uint NumberParameters;
//[MarshalAs(UnmanagedType.ByValArray, SizeConst = 15, ArraySubType = UnmanagedType.U4)]
//public readonly uint[] ExceptionInformation;
public unsafe fixed uint ExceptionInformation[15];
}
public enum DebugEventType : int
{
CREATE_PROCESS_DEBUG_EVENT = 3, //Reports a create-process debugging event. The value of u.CreateProcessInfo specifies a CREATE_PROCESS_DEBUG_INFO structure.
CREATE_THREAD_DEBUG_EVENT = 2, //Reports a create-thread debugging event. The value of u.CreateThread specifies a CREATE_THREAD_DEBUG_INFO structure.
EXCEPTION_DEBUG_EVENT = 1, //Reports an exception debugging event. The value of u.Exception specifies an EXCEPTION_DEBUG_INFO structure.
EXIT_PROCESS_DEBUG_EVENT = 5, //Reports an exit-process debugging event. The value of u.ExitProcess specifies an EXIT_PROCESS_DEBUG_INFO structure.
EXIT_THREAD_DEBUG_EVENT = 4, //Reports an exit-thread debugging event. The value of u.ExitThread specifies an EXIT_THREAD_DEBUG_INFO structure.
LOAD_DLL_DEBUG_EVENT = 6, //Reports a load-dynamic-link-library (DLL) debugging event. The value of u.LoadDll specifies a LOAD_DLL_DEBUG_INFO structure.
OUTPUT_DEBUG_STRING_EVENT = 8, //Reports an output-debugging-string debugging event. The value of u.DebugString specifies an OUTPUT_DEBUG_STRING_INFO structure.
RIP_EVENT = 9, //Reports a RIP-debugging event (system debugging error). The value of u.RipInfo specifies a RIP_INFO structure.
UNLOAD_DLL_DEBUG_EVENT = 7, //Reports an unload-DLL debugging event. The value of u.UnloadDll specifies an UNLOAD_DLL_DEBUG_INFO structure.
}
[StructLayout(LayoutKind.Sequential)]
public struct CONTEXT
{
public uint ContextFlags;
public uint Dr0;
public uint Dr1;
public uint Dr2;
public uint Dr3;
public uint Dr6;
public uint Dr7;
public FLOATING_SAVE_AREA FloatSave;
public uint SegGs;
public uint SegFs;
public uint SegEs;
public uint SegDs;
public uint Edi;
public uint Esi;
public uint Ebx;
public uint Edx;
public uint Ecx;
public uint Eax;
public uint Ebp;
public uint Eip;
public uint SegCs;
public uint EFlags;
public uint Esp;
public uint SegSs;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 512)]
public byte[] ExtendedRegisters;
}
public enum CONTEXT_FLAGS : uint
{
CONTEXT_i386 = 0x10000,
CONTEXT_i486 = 0x10000,
CONTEXT_CONTROL = CONTEXT_i386 | 0x01,
CONTEXT_INTEGER = CONTEXT_i386 | 0x02,
CONTEXT_SEGMENTS = CONTEXT_i386 | 0x04,
CONTEXT_FLOATING_POINT = CONTEXT_i386 | 0x08,
CONTEXT_DEBUG_REGISTERS = CONTEXT_i386 | 0x10,
CONTEXT_EXTENDED_REGISTERS = CONTEXT_i386 | 0x20,
CONTEXT_FULL = CONTEXT_CONTROL | CONTEXT_INTEGER | CONTEXT_SEGMENTS,
CONTEXT_ALL = CONTEXT_CONTROL | CONTEXT_INTEGER | CONTEXT_SEGMENTS | CONTEXT_FLOATING_POINT | CONTEXT_DEBUG_REGISTERS | CONTEXT_EXTENDED_REGISTERS
}
[Flags]
public enum ThreadAccessFlags : int
{
TERMINATE = 0x0001,
SUSPEND_RESUME = 0x0002,
GET_CONTEXT = 0x0008,
SET_CONTEXT = 0x0010,
SET_INFORMATION = 0x0020,
QUERY_INFORMATION = 0x0040,
SET_THREAD_TOKEN = 0x0080,
IMPERSONATE = 0x0100,
DIRECT_IMPERSONATION = 0x0200
}
[StructLayout(LayoutKind.Sequential)]
public struct FLOATING_SAVE_AREA
{
public uint ControlWord;
public uint StatusWord;
public uint TagWord;
public uint ErrorOffset;
public uint ErrorSelector;
public uint DataOffset;
public uint DataSelector;
// missing some stuff
public uint Cr0NpxState;
}
[DllImport("kernel32.dll")]
private static extern int VirtualQueryEx(IntPtr hProcess, IntPtr lpAddress, out MEMORY_BASIC_INFORMATION lpBuffer, int dwLength);
[DllImport("kernel32.dll")]
public static extern bool ReadProcessMemory(IntPtr hProcess, int lpBaseAddress, byte[] buffer, int size,
int lpNumberOfBytesRead);
But for some reason it never hits the evt.Exception.ExceptionRecord.ExceptionAddress == address
I'm pretty new to memory reading and have a hard time figuring out what is wrong with the code above.
EDIT: Also if I uncomment the context.Dr7 = 0x00000001; the application that I'm trying to read crashes.

The theory:
You want to attach to a process, put a breakpoint in it, register an event in the debugger application and wait for that event. You need to put the breakpoint at the address you give as parameter, and by reading the context you should be able to see the content of EDX. This seems reasonable, like a human developer would do it.
The implementation:
Looking at your implementation, the method you try to use to put the breakpoint at the address seems suspicious. Reading here I understand that trying to set the context on a running thread might give unpredictable results. You may also lack the sufficient permissions. Assuming you have the permissions, try stopping the threads before setting the context.
The other problem I predict is that the thread you want to stop and debug has a context that needs to be altered as less as possible. I think that you should first stop the thread, read it's context, change the Dr0 flag (assuming that's where you set the breakpoint) and then set the context with all the other register information unaltered.
Without that, I think that you basically change the execution of the program and I have a strong feeling that some of those registers are read only.
These are 2 things you need to consider. Hope it helps.
Debugging:
If that does not work, you need to add a function that uses GetLastError() to see why the functions fail (I'm suspecting SetThreadContext() will be the one causing problems in the first time).
You must also check that the Context structure is defined correctly and all the members have the same order has the ones defined here. The structure has to be aligned by the C# code exactly as in the unmanaged code.
Also please check if you are running on a 64bit OS. The context of a thread in a 64bit OS is different than for 32bit. Registers are extended for 64b and so on. If you use a 64bit OS, the context structure needs to be redefined. Same goes if you use an ARM machine.

Your answer seems to be hiding in one of the comments you transplanted with the other code... The functions to set and get thread context require a handle to a thread (probably opened with OpenThread using appropriate permissions, including at least get/set context). Instead, you're apparently passing a casted process ID. You should consider checking the return value for success, it would've probably helped identify your problem.

C# method's type signature is not pinvoke compatible

I want to use API from Omron V4KU, the documentation described like this :
Original c# code :
const string DllLocation = #"..\..\Libs\Omron\OMCR.dll";
[DllImport(DllLocation)]
public static extern LPCOMCR OMCR_OpenDevice(string lpcszDevice, LPCOMCR_OPTION lpcOption);
public void Start()
{
var lpcOption = new LPCOMCR_OPTION();
var result = OMCR_OpenDevice(null, lpcOption); // error method's type signature is not pinvoke compatible
}
[StructLayout(LayoutKind.Sequential)]
public struct LPCOMCR
{
public string lpcszDevice;
public IntPtr hDevice;
public uint lpcDevice;
}
[StructLayout(LayoutKind.Sequential)]
public struct LPCOMCR_OPTION
{
public uint dwReserved0;
public uint dwReserved1;
public uint dwReserved2;
public uint dwReserved3;
}
if I missed or wrong in writing code?
sorry, my english is bad. thanks for help.

Start by defining the union structure correctly:
// OMCR_OPTION.COM
[StructLayout(LayoutKind.Sequential)]
public struct OmcrCom
{
public IntPtr Reserved0;
public uint BaudRate;
public uint Reserved1;
public uint Reserved2;
public uint Reserved3;
public IntPtr Reserved1;
public IntPtr Reserved2;
}
// OMCR_OPTION.USB
[StructLayout(LayoutKind.Sequential)]
public struct OmcrUsb
{
public uint Reserved0;
public uint Reserved1;
public uint Reserved2;
public uint Reserved3;
}
// OMCR_OPTION (union of COM and USB)
[StructLayout(LayoutKind.Explicit)]
public struct OmcrOptions
{
[FieldOffset(0)]
public OmcrCom Com;
[FieldOffset(0)]
public OmcrUsb Usb;
}
// OMCR
[StructLayout(LayoutKind.Sequential)]
public struct OmcrDevice
{
public string Device;
public IntPtr DeviceHandle;
public IntPtr DevicePointer;
}
[DllImport(dllName: DllLocation, EntryPoint = "OMCR_OpenDevice"]
public static extern IntPtr OmcrOpenDevice(string type, ref OmcrOptions options);
And then call the method, something like:
var options = new OmcrOptions();
options.Com.BaudRate = 115200; // or whatever you need to set
var type = "COM"; // is this USB/COM? not sure
OmcrDevice device;
var devicePtr = OmcrOpenDevice(type, ref options);
if (devicePtr == IntPtr.Zero)
device = (OmcrDevice)Marshal.PtrToStructure(devicePtr, typeof(OmcrDevice));

Well, for one, the documentation is asking you to pass LPCOMCR_OPTION as a pointer - you're passing it as a value. Using ref should help. There's another problem, though, and that's the return value - again, you're trying to interpret it as a value, while the docs say it's a pointer. However, this is a lot trickier than the first error - as far as I'm aware, your only options are using a C++/CLI interop library, or expecting IntPtr as a return value. In any case, you need to handle proper deallocation of the memory you get this way.

You need to do it like this:
[StructLayout(LayoutKind.Sequential)]
public struct OMCR
{
[MarshalAs(UnmanagedType.LPStr)]
public string lpcszDevice;
public IntPtr hDevice;
public IntPtr lpcDevice;
}
[StructLayout(LayoutKind.Sequential)]
public struct OMCR_OPTION
{
public uint dwReserved0;
public uint dwReserved1;
public uint dwReserved2;
public uint dwReserved3;
}
[DllImport(DllLocation, CallingConvention = CallingConvention.???,
SetLastError = true)]
public static extern IntPtr OMCR_OpenDevice(string lpcszDevice,
ref OMCR_OPTION lpcOption);
You need to replace CallingConvention.??? with the appropriate calling convention. We cannot tell from the question what that is. You will have to find out by reading the header file.
The return value is a pointer to OMCR. You need to hold on to this pointer and pass it to OMCR_CloseDevice when you are finished with it.
In order to obtain an OMCR value you would do the following:
OMCR_OPTION Option = new OMCR_OPTION(); // not sure how to initialize this
IntPtr DevicePtr = OMCR_OpenDevice(DeviceType, ref Option);
if (DevicePtr == IntPtr.Zero)
throw new Win32Exception();
OMCR Device = (OMCR)Marshal.PtrToStructure(DevicePtr, typeof(OMCR));

Getting bad IntPtr from P/Invoked code

First, I apologize if my title isn't technically accurate. I'm not sure exactly what's happening and that describes it about as well as anything.
I am attempting to decode an SSL certificate for a specific use in a program. I p/invoked all of the necessary CryptoAPI functions and structs and in debug mode, everything is working as expected. However, when the program is run in release mode as a service (that's the purpose of this program), I get an access violation when attempting to decode the extensions. In order to make extensions easier to decode, I have created a generic class that can be used to represent any extension. This class contains an object of type TStruct that represents the underlying structure that the extension is based on. It has an EncryptedValue field of type byte[] that will encrypt/decrypt the extension data and place it in the structure as appropriate. The base class is as follows:
public abstract class ExtensionBase<TStruct> : IExtensionBase where TStruct : new()
{
//The underlying struct
protected TStruct objectData = new TStruct();
//The identifier of the struct, ie: szOID_BASIC_CONSTRAINTS
public string Identifier { get; protected set; }
//An enum representing the structure type, ie: X509_BASIC_CONSTRAINTS
public CertStructType StructureType { get; protected set; }
//Determines if the extension is critical
public bool IsCritical { get; protected set; }
//Overridden in any child class to determine if that extension actually contains
//data that should be encoded
public abstract bool HasData { get; }
//Encrypts/decrypts the data from/to the underlying structure
public virtual byte[] EncryptedValue
{
get
{
uint encodedSize = 0;
//PinnedHandle is a class that I wrote to wrap a GCHandle.
//It has an implicit cast to IntPtr that returns GCHandle.AddrOfPinnedObject
//The finalizer of the class releases the GCHandle if it is a valid handle
IntPtr dataPtr = new PinnedHandle(objectData);
byte[] retVal = null;
if (StructureType != CertStructType.None)
{
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
(uint)StructureType,
dataPtr,
0,
IntPtr.Zero,
null,
ref encodedSize))
throw new Win32Exception();
retVal = new byte[encodedSize];
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
(uint)StructureType,
dataPtr,
0,
IntPtr.Zero,
retVal,
ref encodedSize))
throw new Win32Exception();
}
else
{
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
Identifier,
dataPtr,
0,
IntPtr.Zero,
null,
ref encodedSize))
throw new Win32Exception();
retVal = new byte[encodedSize];
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
Identifier,
dataPtr,
0,
IntPtr.Zero,
retVal,
ref encodedSize))
throw new Win32Exception();
}
return retVal;
}
set
{
uint decodedSize = 0;
IntPtr decodedData = IntPtr.Zero;
if(StructureType != CertStructType.None)
decodedData = Crypt32.CryptDecodeObjectEx(StructureType, value);
else
decodedData = Crypt32.CryptDecodeObjectEx(Identifier, value);
TStruct data = (TStruct)Marshal.PtrToStructure(decodedData, typeof(TStruct));
objectData = data;
Marshal.FreeHGlobal(decodedData);
}
}
public ExtensionBase(string id)
{
Identifier = id;
StructureType = CertStructType.None;
}
public ExtensionBase(string id, CertStructType structType)
{
Identifier = id;
StructureType = structType;
}
}
One of the child classes that is giving me problems is the CertKeyUsage class which uses a CRYPT_BIT_BLOB struct to represent its data:
public class CertKeyUsage : ExtensionBase<CertKeyUsageFlags, CRYPT_BIT_BLOB>
{
public override bool HasData
{
get { return Value != CertKeyUsageFlags.None; }
}
public override unsafe byte[] EncryptedValue
{
get
{
CertKeyUsageFlags flags = Value;
objectData.cbData = 2;
objectData.cUnusedBits = 0;
objectData.pbData = new IntPtr(&flags);
return base.EncryptedValue;
}
set
{
try
{
//The following code was taken directly from Microsoft's implementation
//of X509Certificate
base.EncryptedValue = value;
if (objectData.cbData > 4)
objectData.cbData = 4;
byte[] keyUsage = new byte[4];
//This if statement returns true, and the following Marshal.Copy statement
//is where the program crashes with the Access Violation. As it is an unmanaged
//exception, the try/catch block doesn't do anything and the app dies
if (objectData.pbData != IntPtr.Zero)
Marshal.Copy(objectData.pbData, keyUsage, 0, (int) objectData.cbData);
Value = (CertKeyUsageFlags) BitConverter.ToUInt32(keyUsage, 0);
}
catch
{
}
}
}
public CertKeyUsage()
: base(CertOid.szOID_KEY_USAGE, CertStructType.X509KeyUsage)
{
IsCritical = true;
}
}
What about this code would be different from debug to release that would cause an access violation at the spot noted above? What could I do differently to get it working properly. This particular extension isn't critical and I could just skip over it, but after commenting out the code above that is causing the exception, another of the extensions will crash. I can't comment out all extensions, and the fact that it's moving to a different location tells me that there is some underlying problem with my code that I'm missing. Any help or suggestions would be greatly appreciated.
These are the p/invoked functions I'm calling:
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptEncodeObjectEx(uint certEncodingType,
[MarshalAs(UnmanagedType.LPStr)]
string structType,
IntPtr structInfo,
uint flags,
IntPtr encodePara,
byte[] encodedData,
[In, Out] ref uint encodedSize);
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptEncodeObjectEx(uint certEncodingType,
uint structType,
IntPtr structInfo,
uint flags,
IntPtr encodePara,
byte[] encodedData,
[In, Out] ref uint encodedSize);
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptDecodeObjectEx(uint certEncodingType,
[MarshalAs(UnmanagedType.LPStr)]
string structType,
byte[] encodedData,
uint encodedDataSize,
uint flags,
IntPtr encodePara,
IntPtr decodedData,
[In, Out] ref uint decodedDataSize);
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptDecodeObjectEx(uint certEncodingType,
uint structType,
byte[] encodedData,
uint encodedDataSize,
uint flags,
IntPtr encodePara,
IntPtr decodedData,
[In, Out] ref uint decodedDataSize);
And the function that wraps CryptDecodeObjectEx:
public static IntPtr CryptDecodeObjectEx(string structType, byte[] encodedData)
{
uint encodedSize = (uint)encodedData.Length;
uint decodedSize = 0;
IntPtr decodedData = IntPtr.Zero;
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
decodedData = Marshal.AllocHGlobal((int)decodedSize);
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
return decodedData;
}
public static IntPtr CryptDecodeObjectEx(uint structType, byte[] encodedData)
{
uint encodedSize = (uint)encodedData.Length;
uint decodedSize = 0;
IntPtr decodedData = IntPtr.Zero;
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
decodedData = Marshal.AllocHGlobal((int)decodedSize);
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
return decodedData;
}
On a whim, I decided to compile the program as X86 rather than AnyCPU and everything works as expected. This leads me to believe that the problem is actually in this section of code:
public static IntPtr Next<T>(this IntPtr val)
{
T retVal = (T)Marshal.PtrToStructure(val, typeof(T));
if (Environment.Is64BitProcess)
return (IntPtr)((long)val + Marshal.SizeOf(retVal));
return (IntPtr)((int)val + Marshal.SizeOf(retVal));
}
This is the code that I use to enumerate through a pointer to an array of structures. The structure type is passed in and an IntPtr to the next structure in the array should be returned. The 32 bit version is working just as it should, but apparently my logic in the 64 bit version is somewhat lacking.