Develop Reference

Calling a void function (in win32) from C# with parameters

I have the following code in win32, which sets a hook in a target application.
void InstallHook(DWORD ThreadId)
{
g_hHook = SetWindowsHookEx(WH_CALLWNDPROC, CallWndProc, g_hInstDll, ThreadId);
}
I wish to call this function from C# (.net).
I have this so far:
[DllImport("TheHookDll.dll")]
public extern static void InstallHook(UInt32 ThreadId);
[DllImport("user32.dll")]
static extern uint GetWindowThreadProcessId(IntPtr hWnd, IntPtr ProcessId);
[DllImport("user32.dll", SetLastError = true)]
static extern IntPtr FindWindow(string lpClassName, string lpWindowName);
and I call it like this:
IntPtr hWnd = FindWindow(null, "MyTargetAppWindowTitle");
UInt32 threadID = GetWindowThreadProcessId(hWnd, IntPtr.Zero);
InstallHook(threadID);
This gives me the handle(hWnd) for the target, and the threadID which is used in the InstallHook function in win32. (It’s just decimals instead of hex)
But I get this error messages:
PInvokeStackImbalance was detected Message: A call to PInvoke function
'TheOperator!TheOperator.Form1::InstallHook' has unbalanced the stack.
This is likely because the managed PInvoke signature does not match
the unmanaged target signature. Check that the calling convention and
parameters of the PInvoke signature match the target unmanaged
signature.
I have tried to change the Calling Convention in my dll file (configuration properties -> C/C++ -> All Options -> Calling Convention) from __cdel to __stdcall, but without any luck. (same error)
What am I doing wrong?
I have changed the DWORD to UInt32, because c# do not support DWORD etc. But is that the right way to do it?
Any hints?

Define your PInvoke like this:
[DllImport("TheHookDll.dll", CallingConvention = CallingConvention.Cdecl)]
public extern static void InstallHook(UInt32 ThreadId);
Reason being that when you call this specific function, the caller needs to clean the stack. Without explicitly specifying this in your PInvoke signature, the runtime won't clean the stack and therefore it'll mess the stack up resulting in the error message you've been seeing.
If you don't specify a CallingConvention explicitly, the runtime assumes the function you are trying to call is an StdCall, in which the callee cleans the stack. That's not the case, and the stack will be left all messed up and dirty.
Other than that your signature looks correct; a DWORD is indeed a uint or UInt32 in C#. If that one gets you into trouble, you could try decorating it with the MarshalAs attribute and have the Marshal return it as unmanaged type U8.

LoadLibrary not failing on missing DLL: why?

I'm sure I'm missing something obvious in the documentation. I'm calling LoadLibrary from C# and passing in a DLL that isn't there. I'm getting back IntPtr.Zero as I would expect, but when I call Marshal.GetLastWin32Error I always get 0.
Here's the sample.
class Program {
[DllImport("kernel32.dll")]
private static extern IntPtr LoadLibrary(string dllToLoad);
[DllImport("kernel32.dll")]
private static extern bool FreeLibrary(IntPtr hModule);
static void Main(string[] args) {
IntPtr pDll = LoadLibrary(#"c:\NotThere.dll");
int err = Marshal.GetLastWin32Error();
Console.WriteLine(err);
Console.ReadLine();
}
}
I'm adding code like this to a program that's already failing to load a DLL for no apparent reason. Any idea why I'm not getting an error message?

For errors to get logged so that they can be read by Marshal.GetLastWin32Error(), you need to have SetLastError=true on the DllImport attribute:
[DllImport("kernel32.dll", SetLastError=true)]
private static extern IntPtr LoadLibrary(string dllToLoad);

Your code doesn't set the DllImport.SetLastError attribute flag, which might be why GetLastWin32Error isn't returning anything:
[DllImport("kernel32.dll", SetLastError=true)]
private static extern IntPtr LoadLibrary(string dllToLoad);
To diagnose why modules aren't being loaded you should check Fusion logs (if the module is a managed assembly), and / or dependency walker.
Bear in mind that the load error might be caused because a dependent module could not be loaded (e.g. the VC9 redistributable, on which all C++ modules compiled with the VS2008 compiler depend on), and so even if the module is architecture compatible and located in a searchable directory (such as the application or system directory), the module may still fail to load - Dependency walker is very good at highlighting these sorts of issues.

You have to declare in your DllImport attribute that you want the last error to be captured using the SetLastError field. For example:
[DllImport("kernel32.dll", SetLastError=true)]
private static extern IntPtr LoadLibrary(string dllToLoad);
[DllImport("kernel32.dll", SetLastError=true)]
private static extern bool FreeLibrary(IntPtr hModule);

Use the following DllImport:
[DllImport("kernel32.dll", SetLastError = true)]
private static extern IntPtr LoadLibrary(string dllToLoad);

try to add this attribute:
[DllImportAttribute("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
from msdn:
Returns the error code returned by the last unmanaged function that was called using platform invoke that has the DllImportAttribute.SetLastError flag set.

accessing functions or methods of the non .net dlls

how to access the methods of non .net dll in .net in c#? which class is used to access the dll methods.

You do this with P/Invoke. An excellent resource is pinvoke.net.
A simple example would be GetTickCount:
[DllImport("kernel32.dll")]
static extern uint GetTickCount();
Something more complex, MessageBox:
[DllImport("user32.dll", CharSet=CharSet.Auto)]
public static extern int MessageBox(IntPtr hWnd, String text, String caption, int options);

Difference between SafeProcessHandle and IntPtr for an API call

While working with WinAPI, I decided to implement a call to GetProcessAfinityMask in my C# application. However, I've seen two different signatures for this function.
One of them uses SafeProcessHandle for the handle:
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
public static extern bool GetProcessAffinityMask(SafeProcessHandle handle, out IntPtr processMask, out IntPtr systemMask);
The other possible version (on P/Invoke) uses IntPtr:
[DllImport("kernel32.dll",SetLastError = true)]
static extern bool GetProcessAffinityMask(IntPtr hProcess,
out UIntPtr lpProcessAffinityMask, out UIntPtr lpSystemAffinityMask);
Given that both functions return the same values, what is the difference between passing a SafeProcessHandle instance or IntPtr?

Safe Handles and Critical Finalization MSDN Article is describing this difference.

How do I get the handle of a console application's window

Can someone tell me how to get the handle of a Windows console application in C#? In a Windows Forms application, I would normally try this.Handle.

Not sure it works, but you can try that :
IntPtr handle = Process.GetCurrentProcess().MainWindowHandle;

The aforementioned Process.MainWindowHandle method only works for the process that started the console
The FindWindowByCaption method is inherently unreliable
Here's a robust way to do this:
The related functions from the Console Win32 API are:
[DllImport("kernel32.dll", SetLastError = true)]
static extern bool AttachConsole(uint dwProcessId);
[DllImport("kernel32.dll")]
static extern IntPtr GetConsoleWindow();
[DllImport("kernel32.dll", SetLastError=true, ExactSpelling=true)]
static extern bool FreeConsole();
For the console the current process is attached to, just GetConsoleWindow() is enough
For the console another process is attached to, attach to it as well with AttachConsole, call GetConsoleWindow, them immediately detach with FreeConsole.
For the extra cautious, register a console event handler before attaching (and unregister it after detaching) as well so you don't accidentally get terminated if a console event happens in the tiny time frame you're attached to the console:
[DllImport("kernel32.dll")]
static extern bool SetConsoleCtrlHandler(ConsoleCtrlDelegate HandlerRoutine,
bool Add);
delegate Boolean ConsoleCtrlDelegate(CtrlTypes CtrlType);
enum CtrlTypes : uint {
CTRL_C_EVENT = 0,
CTRL_BREAK_EVENT,
CTRL_CLOSE_EVENT,
CTRL_LOGOFF_EVENT = 5,
CTRL_SHUTDOWN_EVENT
}
bool is_attached=false;
ConsoleCtrlDelegate ConsoleCtrlDelegateDetach = delegate(CtrlType) {
if (is_attached = !FreeConsole())
Trace.Error('FreeConsole on ' + CtrlType + ': ' + new Win32Exception());
return true;
};
Making changes to the current process just to read something is rather ugly (when this is a console process, this gets really ugly since it requires a helper process to avoid terminating the current console). Nevertheless, further investigation shows that there's no other way short of injecting into the csrss process or the target process.
Console correspondence information is located in and managed by csrss.exe (or a multitude of those, one for each session, since Vista), so it can't be retrieved with the likes of ReadProcessMemory. All that csrss exposes is the CSRSS LPC API. There's only one relevant function in the full API list, SrvGetConsoleWindow. And it doesn't accept a PID but determines that of the calling party as seen in an alternative implementation or the function's disassembly in winsrv.dll.

Try this:
[DllImport("user32.dll", EntryPoint = "FindWindow", SetLastError = true)]
public static extern IntPtr FindWindowByCaption(IntPtr zeroOnly, string lpWindowName);
…
Console.Title = "Test";
…
IntPtr handle = FindWindowByCaption(IntPtr.Zero, Console.Title);

I've just solved this problem for myself (unfortunately before seeing Thomas's answer which is much quicker). Well, here's another way for those who are not satisfied with his answer. I'm writing this answer because I want to provide another answer + a better way to design the Program class if you're treating your console as a Window. Let's begin with that design:
I've kind of changed the default style of the Program class. I've actually made it into a class that has a program in it, and not just one method which represent it and uses other classes for content. (If you don't know what I mean, not important).
The reason I had to do it is because I wanted to write the following event handler:
private void CatchUnhandled(Object sender, UnhandledExceptionEventArgs e)
{
var exception = e.ExceptionObject as Exception;
MessageBox.Show(this, exception.Message, "Error"); // Check out 1st arg.
}
It overloads this method MessageBox.Show(IWin32Window, String, String).
Because Console doesn't implement IWin32Window, I had to implement it myself, of course, in order to just call this in the 1st argument.
Here is the implementation of it and everything else:
Note: this code is copy-pasted from my application, you can feel free to change the access modifiers
Program Class Declaration:
internal class Program : IWin32Window
{
...
}
IWin32Window Implementation:
public IntPtr Handle
{
get { return NativeMethods.GetConsoleWindow(); }
}
It uses the following class:
internal static class NativeMethods
{
[DllImport("kernel32.dll")]
internal static extern IntPtr GetConsoleWindow();
}
Now, the problem is that you can't actually call this in Main, being a static method, so whatever was in Main I've moved to a new method named Start and all Main is doing is creating a new Program and calling Start.
private static void Main()
{
new Program().Start();
}
private void Start()
{
AppDomain.CurrentDomain.UnhandledException += CatchUnhandled;
throw new Exception();
}
The result was, of course, a message-box with my console's window as an owner.
Using this method for a message-box, is of course only one application of this method.

I don't think there is such a thing. The console window is not accessible to the application. You MIGHT try to iterate the process list looking for your own process name. The Process class IIRC contains a property for the program's main window handle, which might be the console window for console applications - which I'm not sure of.

One more version in VB.Net on how to show MessageBox on top of console window
Imports System.Runtime.InteropServices
Imports System.Windows.Forms
Friend Module NativeMethods
<DllImport("kernel32.dll")> Friend Function GetConsoleWindow() As IntPtr
End Function
End Module
NotInheritable Class WndProxy
Implements IWin32Window
ReadOnly Property Handle As IntPtr Implements IWin32Window.Handle
Sub New(_hwnd As IntPtr)
Handle = _hwnd
End Sub
End Class
Module Module1
Sub Main()
' using MainWindowHandle
Dim w = New WndProxy(Process.GetCurrentProcess().MainWindowHandle)
MessageBox.Show(w, "Hi")
' using GetConsoleWindow api
Dim s = New WndProxy(NativeMethods.GetConsoleWindow)
MessageBox.Show(s, "Hi")
End Sub
End Module

In a console application which streamed diagostics to the console, and for which I wanted to disable mouse input, I tried
GetConsoleWindow(),
Process.GetCurrentProcess().MainWindowHandle, and FindWindowByCaption(IntPtr.Zero, Console.Title).
Each of these returned the same non-zero handle, but when I tried to use that handle in SetConsoleMode it threw a "Invalid Handle" exception. Finally I tried SetConsoleMode(GetStdHandle(STD_INPUT_HANDLE), mode | ENABLE_EXTENDED_FLAGS)) with STD_INPUT_HANDLE defined as -10, and it worked. MS's documentation suggests that handles to consoles may be reassigned, and I am not comfortable or happy with this solution, but so far it is the only thing I've found that allows me to disable quick edit mode programmatically. GetStdHandle(STD_INPUT_HANDLE) returns '3', the other calls return a 7 digit value that varies each time the program is run.