Develop Reference

Pass pointer to a pointer in Go DLL Syscall

If I have the following C# DllImport which is importing a non-C# DLL and I want to port it over to Go, how would I go about doing that?
[DllImport("my.dll", EntryPoint = "Greet", CallingConvention = CallingConvention.Cdecl)]
public static extern int Greet(IntPtr name, ref IntPtr greetings);
I've run into problems figuring out how to pass a pointer to a pointer which is needed for the greetings parameter (I assume since the type is ref IntPtr, I'm not that familiar at all with C#). The dll function will populate the memory pointed to by the pointer that I provide which I'll use in subsequent syscalls. Here's what I've got so far,
package main
import (
"fmt"
"syscall"
"unsafe"
)
var (
MyDll = syscall.MustLoadDLL("my.dll")
greet = MyDll.MustFindProc("Greet")
)
func Greet(name string) error {
nameP, err := syscall.UTF16PtrFromString(name)
if err != nil {
return err
}
// I need to provide a pointer to a pointer for greetings. How can I allocate some memory here
// then pass a pointer to its pointer? I tried this: create a handle with a zero-value, then
// take a pointer to it, then pass a pointer to the pointer as the second parameter to Call but
// the pointer becomes nil after the Call.
handle := syscall.Handle(0)
handleP := &handle
r1, _, _ := greet.Call(uintptr(unsafe.Pointer(nameP)), uintptr(unsafe.Pointer(&handleP)))
if r1 == 0 {
return fmt.Errorf("there was an error")
}
return nil
}
I'm open to any and all suggestions including links and resources that might help me get a better grasp on this syscall and unsafe stuff. Thanks!

Firstly, it would help if you could show how the C# Greet method is used. A method in isolation its quite hard to understand especially when the parameter is the equivalent of void ** which means anything can go in.
TL;DR
ref IntPtr is probably just a **struct{} where you don't have to allocate any struct. The library will simply manage the memory for you. You just need to give it a pointer to "*MyStruct" so that it can change you "*MyStruct" to actually point to the internal resource.
REF
The C# ref keyword is quite well explained in the docs. Basically it allows a pass by reference for any type.
The following declaration in C#
void Increment(ref value) { ... }
int counter = 10;
Increment(ref counter);
// counter is now 11
Increment(counter); // won't compile must be passed with 'ref'
Increment(null); // won't compile
is equivalent to C++
void Increment(int& value) { ... }
int counter;
Increment(counter);
// counter is now 11
Increment(null); // won't compile
A reference which should not be null.
IntPtr
IntPtr is usually used to represent pointers and allows for interop between native and CLR (C#) programs.
If a C program has the following signature
void Increment(int* value);
a C# program could call it in one of a few ways
[DllImport("example.dll")]
static unsafe extern void Increment(int* value); // this way allows null to be passed in
unsafe {
int counter = 10;
Increment(&10);
}
,
[DllImport("example.dll")]
static extern void Increment(ref int value);
int counter = 10;
Increment(ref counter);
If a C program has the following signature
void AllocateStruct(struct MyStruct** ppStruct);
void IncrementStruct(struct MyStruct* pStruct);
then
[DllImport("example.dll")]
static extern void AllocateStruct(ref IntPtr ppStruct);
// static unsafe extern void AllocateStruct(MyStruct** ppStruct)
[DllImport("example.dll")]
static extern void IncrementStruct(IntPtr pStruct);
// static unsafe extern void IncrementStruct(MyStruct* pStruct);
IntPtr pMyStruct;
AllocateStruct(ref pMyStruct);
IncrementStruct(pMyStruct);
// Free My Struct
// If you need to get inside the struct then
// MyStruct myStruct = Marshal.StructureToPtr<MyStruct>(pMyStruct)
// Often you don't (need to) or (should not) manipulate the struct directly so keeping it as IntPtr is perfectly acceptable.
From the example above you can see MyStruct is more of a token/reference than anything else. ref IntPtr allows you to pass a reference to the location which you will use to store your token/reference after the library allocates it on your behalf. Then all the other methods will usually just use the reference IntPtr to perform subsequent manipulations on it. Basically Object Orientated Programming without classes.
"Real" Life Example with ref IntPtr
It is a bit quick and dirty and error handling leave a lot to be desired.
It shows the C, C# and Go versions which call the same GetSecurityInfo
and LookupAccountSid Win32 library functions.
The only real use case I can find for ref IntPtr is type**/void** so that the library can allocate the memory for you or give you a pointer to memory it has already allocated.
C
#include <stdio.h>
#include <windows.h>
#include <tchar.h>
#include "accctrl.h"
#include "aclapi.h"
#pragma comment(lib, "advapi32.lib")
int _tmain(int argc, _TCHAR* argv[])
{
// --- get the executing program's file path and handle -----
LPTSTR executablePath = argv[0];
_tprintf(TEXT("Opening File %s\n"), executablePath);
HANDLE hFile = CreateFile(
executablePath,
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (hFile == INVALID_HANDLE_VALUE) return EXIT_FAILURE;
// -------------------------------------------------
// --------- Get the owner SID of the file ---------
PSID pSidOwner = NULL;
PSECURITY_DESCRIPTOR pSD = NULL;
DWORD dwRtnCode = GetSecurityInfo(
hFile,
SE_FILE_OBJECT,
OWNER_SECURITY_INFORMATION,
&pSidOwner,
NULL,
NULL,
NULL,
&pSD);
if (dwRtnCode != ERROR_SUCCESS) return EXIT_FAILURE;
// -------------------------------------------------
// -------
TCHAR AcctName[MAX_PATH];
DWORD dwAcctName = MAX_PATH;
TCHAR DomainName[MAX_PATH];
DWORD dwDomainName = MAX_PATH;
SID_NAME_USE eUse = SidTypeUnknown;
BOOL bRtnBool = LookupAccountSid(
NULL, // local computer
pSidOwner,
&AcctName,
&dwAcctName,
DomainName,
&dwDomainName,
&eUse);
if (bRtnBool == FALSE) return EXIT_FAILURE;
_tprintf(TEXT("Account Owner = %s\n"), AcctName);
_tprintf(TEXT("Account Owner's Domain = %s\n"), DomainName);
return 0;
}
C#
using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
public class Example
{
[DllImport("advapi32.dll", SetLastError = true, CallingConvention = CallingConvention.Winapi)]
static extern uint GetSecurityInfo(
IntPtr handle,
uint ObjectType,
uint SecurityInfo,
ref IntPtr ppsidOwner, // <-- HERE
IntPtr ppsidGroup, // bit hacky (in safe C# you must "pass a reference" in C you can pass a pointer to a pointer or null)
IntPtr ppDacl,
IntPtr ppSacl,
ref IntPtr ppSecurityDescriptor // <-- HERE
);
[DllImport("advapi32.dll", CharSet = CharSet.Auto, SetLastError = true)]
static extern bool LookupAccountSid(
string lpSystemName,
IntPtr Sid,
StringBuilder lpName,
ref uint cchName,
StringBuilder ReferencedDomainName,
ref uint cchReferencedDomainName,
out uint peUse);
const uint ERROR_SUCCESS = 0;
const uint OWNER_SECURITY_INFORMATION = 0x00000001;
const uint SE_FILE_OBJECT = 1;
public static void Main()
{
// get the executing program's file path and handle
string executablePath = Environment.GetCommandLineArgs().GetValue(0).ToString();
IntPtr hFile = File.Open(executablePath, FileMode.Open, FileAccess.Read, FileShare.Read)
.SafeFileHandle.DangerousGetHandle();
IntPtr pSidOwner = IntPtr.Zero; // some internal struct you shouldn't allocate or modify (acts like a token)
IntPtr pSD = IntPtr.Zero; // some internal struct you shouldn't allocate or modify (acts like a token)
// vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
uint dwRtnCode = GetSecurityInfo(
hFile,
SE_FILE_OBJECT,
OWNER_SECURITY_INFORMATION,
ref pSidOwner, // <-- HERE
IntPtr.Zero,
IntPtr.Zero,
IntPtr.Zero,
ref pSD // <-- HERE
);
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
if (dwRtnCode != ERROR_SUCCESS) throw new InvalidOperationException("GetSecurityInfo Failed");
StringBuilder name = new StringBuilder(50);
uint cchName = (uint)name.Capacity;
StringBuilder domainName = new StringBuilder(50);
uint cchDomainName = (uint)domainName.Capacity;
uint sidUse;
LookupAccountSid(
null,
pSidOwner,
name,
ref cchName,
domainName,
ref cchDomainName,
out sidUse);
Console.WriteLine("Account Owner = {0}", name);
Console.WriteLine("Account Owner's Domain = {0}", domainName);
// PLEASE FREE pSD once done
}
}
Go
my second Go program I have ever written so there are probably some blaring mistakes (other than the lack of error checking)
package main
import (
"fmt"
"syscall"
"unsafe"
"os"
)
var (
advapi32, _ = syscall.LoadLibrary("advapi32.dll")
kernel32, _ = syscall.LoadLibrary("kernel32.dll")
createFileW, _ = syscall.GetProcAddress(kernel32, "CreateFileW")
getSecurityInfo, _ = syscall.GetProcAddress(advapi32, "GetSecurityInfo")
lookupAccountSidW, _ = syscall.GetProcAddress(advapi32, "LookupAccountSidW")
)
type SE_OBJECT_TYPE uint32
const (SE_FILE_OBJECT = 1)
type SECURITY_INFORMATION uint32
const (OWNER_SECURITY_INFORMATION = 0x00000001)
const (
GENERIC_READ = 0x80000000
FILE_SHARE_READ = 0x00000001
OPEN_EXISTING = 0x00000003
FILE_ATTRIBUTE_NORMAL = 0x00000080
)
type Handle uintptr
func CreateFile(
name string,
access uint32,
mode uint32,
sa *uint, // *SecurityAttributes,
createmode uint32,
attrs uint32,
templatefile *uint,
) (handle Handle, err error) {
utf16name, _ := syscall.UTF16PtrFromString(name)
r0, _, _ := syscall.Syscall9(
uintptr(createFileW), 7,
uintptr(unsafe.Pointer(utf16name)),
uintptr(access),
uintptr(mode),
uintptr(unsafe.Pointer(sa)),
uintptr(createmode),
uintptr(attrs),
uintptr(unsafe.Pointer(templatefile)),
0, 0)
handle = Handle(r0)
return
}
func GetSecurityInfo(
handle Handle,
objectType SE_OBJECT_TYPE,
securityInformation SECURITY_INFORMATION,
owner **struct{},
group **struct{},
dacl **struct{},
sacl **struct{},
sd **struct{}, //**SECURITY_DESCRIPTOR,
) (ret error) {
r0, _, _ := syscall.Syscall9(
uintptr(getSecurityInfo), 8,
uintptr(handle),
uintptr(objectType),
uintptr(securityInformation),
uintptr(unsafe.Pointer(owner)),
uintptr(unsafe.Pointer(group)),
uintptr(unsafe.Pointer(dacl)),
uintptr(unsafe.Pointer(sacl)),
uintptr(unsafe.Pointer(sd)),
0)
if r0 != 0 {
ret = syscall.Errno(r0)
}
return
}
func LookupAccountSid(
systemName *uint16,
sid *struct{}, // *SID,
name *uint16,
nameLen *uint32,
refdDomainName *uint16,
refdDomainNameLen *uint32,
use *uint32,
) (err error) {
r, _, e := syscall.Syscall9(
uintptr(lookupAccountSidW), 7,
uintptr(unsafe.Pointer(systemName)),
uintptr(unsafe.Pointer(sid)),
uintptr(unsafe.Pointer(name)),
uintptr(unsafe.Pointer(nameLen)),
uintptr(unsafe.Pointer(refdDomainName)),
uintptr(unsafe.Pointer(refdDomainNameLen)),
uintptr(unsafe.Pointer(use)),
0, 0)
if r == 0 {
err = e
}
return
}
func main() {
defer syscall.FreeLibrary(advapi32)
defer syscall.FreeLibrary(kernel32)
// get the executing program's file path and handle
var hFile, _ = CreateFile(os.Args[0], GENERIC_READ, FILE_SHARE_READ, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nil);
// defer LocalFree(Handle(unsafe.Pointer(pSD))) // PLEASE FREE pSD once done
// vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
var pSD *struct{} //*SECURITY_DESCRIPTOR
var pSidOwner *struct{}
GetSecurityInfo(hFile, SE_FILE_OBJECT, OWNER_SECURITY_INFORMATION, &pSidOwner, nil, nil, nil, &pSD)
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nameLen := uint32(50)
name := make([]uint16, nameLen)
domainLen := uint32(50)
domainName := make([]uint16, domainLen)
var sidUse uint32
LookupAccountSid(nil, pSidOwner, &name[0], &nameLen, &domainName[0], &domainLen, &sidUse)
var n = syscall.UTF16ToString(name)
var dn = syscall.UTF16ToString(domainName)
fmt.Printf("Account Owner = %s\n", n)
fmt.Printf("Account Owner's Domain = %s\n", dn)
}

How to call function in C# from memory like in C++?

My problem is in calling shellcode in C#. I alloc memory in the process, write there my bytes and try to call it with Marshal.GetDelegateFromFunctionPointer(). It seems working, but, only if my function doesn't have parameters. If there's any parameter there then it's value equals {Unable to read memory}. In C++ I just need to typedef func, cast it, and call. How to do this properly in C#? (Shellcode performs jump to another C# func)

First of all in C# you have to declare delegate with all required parameters and calling convension. When you use GetDelegateForFunctionPointer ptr is converted to a delegate that invokes the unmanaged method using the __stdcall calling convention on Windows, or the __cdecl calling convention on Linux and macOS. You can set the calling convention by applying the UnmanagedFunctionPointerAttribute to the delegate.
Also you need indicate how to marshal parameters between managed and unmanaged code to achieve this you should use MarshalAsAttribute
See remarks section of Marshal.GetDelegateForFunctionPointer
and UnmanagedFunctionPointerAttribute
and MarshalAsAttribute
Example:
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, ExactSpelling = true)]
private static extern IntPtr LoadLibraryW([MarshalAs(UnmanagedType.LPWStr)] string libFileName);
[DllImport("kernel32.dll", CharSet = CharSet.Ansi, ExactSpelling = true)]
private static extern IntPtr GetProcAddress(IntPtr hModule, [MarshalAs(UnmanagedType.LPStr)] string lpProcName);
const int MEM_COMMIT = 0x00001000;
const int PAGE_READWRITE = 0x04;
const int PAGE_EXECUTE_READ = 0x20;
[DllImport("kernel32.dll", ExactSpelling = true)]
private static extern IntPtr VirtualAlloc(IntPtr address, IntPtr size, int allocationType, int protect);
[DllImport("kernel32.dll", ExactSpelling = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool VirtualProtect(IntPtr address, IntPtr size, int newProtect, out int oldProtect);
private delegate int GetTempPath(int nBufferLength, [MarshalAs(UnmanagedType.LPStr)] StringBuilder buffer);
static void Main(string[] args)
{
IntPtr hModule = LoadLibraryW("c:\\windows\\system32\\kernel32.dll");
IntPtr procAddress = GetProcAddress(hModule, "GetTempPathA");
// allocate page with read/write access
IntPtr page = VirtualAlloc(IntPtr.Zero, new IntPtr(4096), MEM_COMMIT, PAGE_READWRITE);
// write to allocated memory
// mov r11, procAddress
Marshal.WriteByte(page, 0, 0x49);
Marshal.WriteByte(page, 1, 0xBB);
Marshal.WriteInt64(page, 2, procAddress.ToInt64());
// jmp r11
Marshal.WriteByte(page, 0xA, 0x41);
Marshal.WriteByte(page, 0xB, 0xFF);
Marshal.WriteByte(page, 0xC, 0xE3);
// protect memory to allow execute code
bool result = VirtualProtect(page, new IntPtr(4096), PAGE_EXECUTE_READ, out int oldProtect);
var getTempPathTrampoline = Marshal.GetDelegateForFunctionPointer<GetTempPath>(page);
StringBuilder builder = new StringBuilder(1024);
getTempPathTrampoline(1024, builder);
Console.WriteLine(builder.ToString());
}
Example (call managed code)
static void Main(string[] args)
{
GetTempPath delegateToManaged = ManagedMethod;
IntPtr managedPtr = Marshal.GetFunctionPointerForDelegate(delegateToManaged);
// allocate page with read/write access
IntPtr page = VirtualAlloc(IntPtr.Zero, new IntPtr(4096), MEM_COMMIT, PAGE_READWRITE);
// write to allocated memory
// mov r11, procAddress
Marshal.WriteByte(page, 0, 0x49);
Marshal.WriteByte(page, 1, 0xBB);
Marshal.WriteInt64(page, 2, managedPtr.ToInt64());
// jmp r11
Marshal.WriteByte(page, 0xA, 0x41);
Marshal.WriteByte(page, 0xB, 0xFF);
Marshal.WriteByte(page, 0xC, 0xE3);
// protect memory to allow execute code
bool result = VirtualProtect(page, new IntPtr(4096), PAGE_EXECUTE_READ, out int oldProtect);
var getTempPathTrampoline = Marshal.GetDelegateForFunctionPointer<GetTempPath>(page);
StringBuilder builder = new StringBuilder(1024);
getTempPathTrampoline(1024, builder);
Console.WriteLine(builder.ToString());
}
public static int ManagedMethod(int nBufferLength, StringBuilder buffer)
{
buffer.Append("Hello World");
return 0;
}

How to call a CPU instruction from C#?

My processor (Intel i7) supports the POPCNT instruction and I would like to call it from my C# application. Is this possible?
I believe I read somewhere that it isn't, but the JIT will invoke it if it finds it available but what function would I have to call that may be substituted with such an instruction?
Popcount is being called millions of times in a loop so I'd like to be able to have this CPU optimization if possible.

You want to play with fire, and here we like to play with fire...
class Program
{
const uint PAGE_EXECUTE_READWRITE = 0x40;
const uint MEM_COMMIT = 0x1000;
[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr VirtualAlloc(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, uint flProtect);
private delegate int IntReturner();
static void Main(string[] args)
{
List<byte> bodyBuilder = new List<byte>();
bodyBuilder.Add(0xb8); // MOV EAX,
bodyBuilder.AddRange(BitConverter.GetBytes(42)); // 42
bodyBuilder.Add(0xc3); // RET
byte[] body = bodyBuilder.ToArray();
IntPtr buf = VirtualAlloc(IntPtr.Zero, (IntPtr)body.Length, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
Marshal.Copy(body, 0, buf, body.Length);
IntReturner ptr = (IntReturner)Marshal.GetDelegateForFunctionPointer(buf, typeof(IntReturner));
Console.WriteLine(ptr());
}
}
(this small example of assembly will simply return 42... I think it's the perfect number for this answer :-) )
In the end the trick is that:
A) You must know the opcodes corresponding to the asm you want to write
B) You use VirtualAlloc to make a page of memory executable
C) In some way you copy your opcodes there
(the code was taken from http://www.cnblogs.com/netact/archive/2013/01/10/2855448.html)
Ok... the other one was as written on the site (minus an error on the uint -> IntPtr dwSize), this one is how it should be written (or at least it's a +1 compared to the original... I would encapsulate everything in a IDisposable class instead of using try... finally)
class Program
{
const uint PAGE_READWRITE = 0x04;
const uint PAGE_EXECUTE = 0x10;
const uint MEM_COMMIT = 0x1000;
const uint MEM_RELEASE = 0x8000;
[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr VirtualAlloc(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, uint flProtect);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool VirtualProtect(IntPtr lpAddress, IntPtr dwSize, uint flAllocationType, out uint lpflOldProtect);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool VirtualFree(IntPtr lpAddress, IntPtr dwSize, uint dwFreeType);
private delegate int IntReturner();
static void Main(string[] args)
{
List<byte> bodyBuilder = new List<byte>();
bodyBuilder.Add(0xb8); // MOV EAX,
bodyBuilder.AddRange(BitConverter.GetBytes(42)); // 42
bodyBuilder.Add(0xc3); // RET
byte[] body = bodyBuilder.ToArray();
IntPtr buf = IntPtr.Zero;
try
{
// We VirtualAlloc body.Length bytes, with R/W access
// Note that from what I've read, MEM_RESERVE is useless
// if the first parameter is IntPtr.Zero
buf = VirtualAlloc(IntPtr.Zero, (IntPtr)body.Length, MEM_COMMIT, PAGE_READWRITE);
if (buf == IntPtr.Zero)
{
throw new Win32Exception();
}
// Copy our instructions in the buf
Marshal.Copy(body, 0, buf, body.Length);
// Change the access of the allocated memory from R/W to Execute
uint oldProtection;
bool result = VirtualProtect(buf, (IntPtr)body.Length, PAGE_EXECUTE, out oldProtection);
if (!result)
{
throw new Win32Exception();
}
// Create a delegate to the "function"
// Sadly we can't use Funct<int>
var fun = (IntReturner)Marshal.GetDelegateForFunctionPointer(buf, typeof(IntReturner));
Console.WriteLine(fun());
}
finally
{
if (buf != IntPtr.Zero)
{
// Free the allocated memory
bool result = VirtualFree(buf, IntPtr.Zero, MEM_RELEASE);
if (!result)
{
throw new Win32Exception();
}
}
}
}
}

C# P/Invoke and array of structs containing byte arrays

I need to invoke a native DLL from C# code. As I am not very familiar with C/C++, I can't figure out how a structure defined in C should be declared in C# so it can be invoked. The problem is that two parameters seems to be an array of structs, which I don't know how to declare this in C# (see last code block):
c++ header file:
typedef enum
{
OK = 0,
//others
} RES
typedef struct
{
unsigned char* pData;
unsigned int length;
} Buffer;
RES SendReceive(uint32 deviceIndex
Buffer* pReq,
Buffer* pResp,
unsigned int* pReceivedLen,
unsigned int* pStatus);
c# declaration:
enum
{
OK = 0,
//others
} RES
struct Buffer
{
public uint Length;
public ??? Data; // <-- I guess it's byte[]
}
[DllImport("somemodule.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern uint SendReceive(
uint hsmIndex,
uint originatorId,
ushort fmNumber,
??? pReq, // <-- should this be ref Buffer[] ?
uint reserved,
??? pResp, // <-- should this be ref Buffer[] ?
ref uint pReceivedLen,
ref uint pFmStatus);
in an equivalent java client, i found that the parameter is not just one Buffer but an array of Buffers. In C# it would look like this:
var pReq = new Buffer[]
{
new Buffer { Data = new byte[] { 1, 0 }, Length = (uint)2 },
new Buffer {Data = requestStream.ToArray(), Length = (uint)requestStream.ToArray().Length },
//according to the header file, the last item must be {NULL, 0}
new Buffer { Data = null, Length = 0 }
};
var pResp = new Buffer[]
{
new Buffer { Data = new byte[0x1000], Length = 0x1000 },
//according to the header file, the last item must be {NULL, 0}
new Buffer { Data = null, Length = 0x0 }
};
This seems strange to me because the extern C method does have a pointer to a Buffer struct (Buffer*) and not a pointer to a Buffer array (Buffer[]*).
How do I need to define the Struct in C# and the parameter types of the extern method?
Any help appreciated, Thanks.

Firstly your struct has the parameters in the wrong order. And the byte array needs to be declared as IntPtr with manual marshalling:
struct Buffer
{
public IntPtr Data;
public uint Length;
}
The p/invoke should be:
[DllImport("MyNativeDll.dll", CallingConvention=CallingConvention.Cdecl)]
static extern RES SendReceive(
uint deviceIndex,
[In] Buffer[] pReq,
[In, Out] Buffer[] pResp,
out uint pReceivedLen,
out uint pStatus
);
The byte array needs to be IntPtr so that the struct is blittable. And that's needed so that the array parameters can be declared as Buffer[].
It's going to be a bit of a pain doing the marshalling of the byte arrays. You'll want to use GCHandle to pin the managed byte arrays, and call AddrOfPinnedObject() to get the address of the pinned array for each struct in your arrays of structs. It will be worth your while writing some helper functions to make that task less painful.

Your method signature in c# should be something like:
[DllImport("MyNativeDll.dll")]
public static extern RES SendReceive (uint32 deviceIndex, ref Buffer pReq, ref Buffer pResp, ref uint pReceivedLen, ref uint pStatus);
See this project, it might hel you in the future so generate native calls from .net
http://clrinterop.codeplex.com/releases/view/14120

Based on the C++ header but without testing, have a look at the following code:
using System;
using System.Runtime.InteropServices;
using System.Text;
namespace WindowsFormsApplication1
{
public class Class1
{
public struct Buffer
{
[MarshalAs(UnmanagedType.LPStr)]
public StringBuilder pData;
public uint length;
}
[DllImport("kernel32.dll", EntryPoint = "LoadLibrary")]
static extern int LoadLibrary(string lpLibFileName);
[DllImport("kernel32.dll", EntryPoint = "GetProcAddress")]
static extern IntPtr GetProcAddress(int hModule, string lpProcName);
[DllImport("kernel32.dll", EntryPoint = "FreeLibrary")]
static extern bool FreeLibrary(int hModule);
[UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
internal delegate IntPtr SendReceive(
uint deviceIndex,
ref Buffer pReq,
ref Buffer pResp,
uint pReceivedLen,
uint pStatus);
public void ExecuteExternalDllFunction()
{
int dll = 0;
try
{
dll = LoadLibrary(#"somemodule.dll");
IntPtr address = GetProcAddress(dll, "SendReceive");
uint deviceIndex = 0;
Buffer pReq = new Buffer() { length = 0, pData = new StringBuilder() };
Buffer pResp = new Buffer() { length = 0, pData = new StringBuilder() };
uint pReceivedLen = 0;
uint pStatus = 0;
if (address != IntPtr.Zero)
{
SendReceive sendReceive = (SendReceive)Marshal.GetDelegateForFunctionPointer(address, typeof(SendReceive));
IntPtr ret = sendReceive(deviceIndex, ref pReq, ref pResp, pReceivedLen, pStatus);
}
}
catch (Exception Ex)
{
//handle exception...
}
finally
{
if (dll > 0)
{
FreeLibrary(dll);
}
}
}
}
}

C# to C++ process with WM_COPYDATA passing struct with strings

From a c# program I want to use WM_COPYDATA with SendMessage to communicate with a legacy c++/cli MFC application.
I want to pass a managed struct containing string objects.
I can find the handle to the c++ application for use with SendMessage fine.
The bit I don't know about is how the struct and it's strings can be marshalled and read at the other end. Especially as it contains non-blittables.
Do people think this is feasible?
I'll continue to work on it, but would apprecite someone who's done this sort of thing telling me if it just isn't going to work.
Here is some demo code if it was a c++/cli program and it's not difficult to get it working.
However, I'd like this to be in a .Net class library so it can easily be re-used.
//Quick demonstation code only, not correctly styled
int WINAPI WinMain(HINSTANCE hInstance,
HINSTANCE hPrevInstance,
LPSTR lpCmdLine,
int nCmdShow)
{
struct MessageInfo
{
int nVersion;
char szTest[ 10 ];
};
MessageInfo sMessageInfo;
sMessageInfo.nVersion = 100;
strcpy( sMessageInfo.szTest, "TEST");
COPYDATASTRUCT CDS;
CDS.dwData = 1; //just for test
CDS.cbData = sizeof( sMessageInfo );
CDS.lpData = &sMessageInfo;
//find running processes and send them a message
//can't just search for "MYAPP.exe" as will be called "MYAPP.exe *32" on a 64bit machine
array<System::Diagnostics::Process^>^allProcesses = System::Diagnostics::Process::GetProcesses();
for each (System::Diagnostics::Process^ targetProcess in allProcesses)
{
if (targetProcess->ProcessName->StartsWith("MYAPP", System::StringComparison::OrdinalIgnoreCase))
{
HWND handle = static_cast<HWND>(targetProcess->MainWindowHandle.ToPointer());
BOOL bReturnValue = SendMessage( handle, WM_COPYDATA, (WPARAM)0, (LPARAM)&CDS ) == TRUE;
}
}
return 0;
}

I have it working.
A simple approach is to serialize the struct to a single string and transfer a string.
The swhistlesoft blog was helpful http://www.swhistlesoft.com/blog/2011/11/19/1636-wm_copydata-with-net-and-c
This may be enough to provide the simple messaging.
The struct can be re-constructed at the other end if necessary.
If a struct with any number of strings is to be marshalled as-is then it must be a fixed size, that's the main thing I wasn't getting.
The
MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst = 9)
basically sets the size to match the c++ size which in our case is a TCHAR szTest[ 9 ];
In order to transfer a .Net struct via WM_COPYDATA from c# to c++(/cli) I had to do as follows:
[System.Runtime.InteropServices.DllImport("user32.dll", CharSet = System.Runtime.InteropServices.CharSet.Auto)]
static extern IntPtr SendMessage(IntPtr hWnd, UInt32 Msg, IntPtr wParam, IntPtr lParam);
[System.Runtime.InteropServices.DllImport("user32.dll", CharSet = System.Runtime.InteropServices.CharSet.Auto)]
static extern bool SetForegroundWindow(IntPtr hWnd);
public static uint WM_COPYDATA = 74;
//from swhistlesoft
public static IntPtr IntPtrAlloc<T>(T param)
{
IntPtr retval = System.Runtime.InteropServices.Marshal.AllocHGlobal(System.Runtime.InteropServices.Marshal.SizeOf(param));
System.Runtime.InteropServices.Marshal.StructureToPtr(param, retval, false);
return (retval);
}
//from swhistlesoft
public static void IntPtrFree(IntPtr preAllocated)
{
if (IntPtr.Zero == preAllocated) throw (new Exception("Go Home"));
System.Runtime.InteropServices.Marshal.FreeHGlobal(preAllocated);
preAllocated = IntPtr.Zero;
}
[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential)]
struct COPYDATASTRUCT
{
public uint dwData;
public int cbData;
public IntPtr lpData;
}
/// <summary>
/// Dot net version of AppInfo structure. Any changes to the structure needs reflecting here.
/// struct must be a fixed size for marshalling to work, hence the SizeConst entries
/// </summary>
[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential, Pack = 1)]
struct AppInfoDotNet
{
public int nVersion;
[System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst = 9)]
public string test;
};
To send a string:
COPYDATASTRUCT cd = new COPYDATASTRUCT();
cd.dwData = 2;
cd.cbData = parameters.Length + 1;
cd.lpData = System.Runtime.InteropServices.Marshal.StringToHGlobalAnsi(parameters);
IntPtr cdBuffer = IntPtrAlloc(cd);
messageReceived = ((int)SendMessage(targetProcess.MainWindowHandle, WM_COPYDATA, IntPtr.Zero, cdBuffer)) != 0;
To receive string in C++:
else if(pCDS->dwData == 2)
{
//copydata message
CString csMessage = (LPCTSTR)pCDS->lpData;
OutputDebugString("Copydata message received: " + csMessage);
}
To send struct:
AppInfoDotNet appInfo = new AppInfoDotNet();
appInfo.test = "a test";
COPYDATASTRUCT cds3;
cds3.dwData = 1;
cds3.cbData = System.Runtime.InteropServices.Marshal.SizeOf(appInfo);
IntPtr structPtr = System.Runtime.InteropServices.Marshal.AllocCoTaskMem(System.Runtime.InteropServices.Marshal.SizeOf(appInfo));
System.Runtime.InteropServices.Marshal.StructureToPtr(appInfo, structPtr, false);
cds3.lpData = structPtr;
IntPtr iPtr = System.Runtime.InteropServices.Marshal.AllocCoTaskMem(System.Runtime.InteropServices.Marshal.SizeOf(cds3));
System.Runtime.InteropServices.Marshal.StructureToPtr(cds3, iPtr, false);
messageReceived = ((int)SendMessage(targetProcess.MainWindowHandle, WM_COPYDATA, IntPtr.Zero, iPtr)) != 0;
System.Runtime.InteropServices.Marshal.FreeCoTaskMem(iPtr);
System.Runtime.InteropServices.Marshal.FreeCoTaskMem(structPtr);
To receive struct in C++:
LRESULT CMainFrame::OnCopyData( WPARAM wParam, LPARAM lParam )
{
LRESULT lResult = FALSE;
COPYDATASTRUCT *pCDS = (COPYDATASTRUCT*)lParam;
//Matching message type for struct
if(pCDS->dwData == 1)
{
AppInfo *pAppInfo = (AppInfo*)pCDS->lpData
lResult = true;
}
Please note this is demo code and needs work in terms of styling, exception handling etc, etc...

From the documentation:
The data being passed must not contain pointers or other references to objects not accessible to the application receiving the data.
So you need to pack your string into COPYDATASTRUCT.lpData. If you have a max length for each string then you can embed it in a fixed length structure
typedef struct tagMYDATA
{
char s1[80];
char s2[120];
} MYDATA;
If you have only one variable length string you can put the string at the end and use a header followed by string data
typedef struct tagMYDATA
{
int value1;
float value2;
int stringLen;
} MYDATAHEADER;
MyCDS.cbData = sizeof(MYDATAHEADER)+(int)stringData.size();
MyCDS.lpData = new BYTE[MyCDS.cbData];
memcpy(MyCDS.lpData,&dataHeader,sizeof*(MYDATAHEADER);
StringCbCopyA (
((BYTE*)MyCDS.lpData)+sizeof*(MYDATAHEADER)
,stringData.size()
,stringData.c_str());
If you have multiple variable length strings you can still use a header and allocate more spaces for every strings plus a double null terminator, or serialize everything into one XML string.