I am investigating a bug that results in key-presses being 'lost' when users are providing rapid keyboard input to our application. It seems to happen randomly. If a user writes e.g. "I am writing this text fast on my keyboard" the actual text that ends up in the application's text box looks something like "Ia wrting thistext fstn mykeybrd."
We have managed to find out that the issue seems to be related to how we integrate the text-input component into our main application. Our main app is a .NET Framework 4.8 WinForms process. The text-input component is running as a .NET 4.8 WPF app in a separate process. The WinForms app is responsible for launching the WPF process, passing the window handle of its own main window as an argument to the new process. Once launched, the WPF process uses the provided window handle to reparent itself as a child window of the WinForms app. This is done using Win32 API calls through P/Invoke.
private void Reparent() {
ChangeStyle(this.myHandle);
Win32.SetParent(this.myHandle, this.hostHandle);
}
private static void ChangeStyle(IntPtr myHandle) {
const int GWL_STYLE = -16;
const uint WS_POPUP = 0x80000000;
const uint WS_CHILD = 0x40000000;
uint style = Win32.GetWindowLong(myHandle, GWL_STYLE);
style |= WS_CHILD;
style &= ~WS_POPUP;
Win32.SetWindowLong(myHandle, GWL_STYLE, style);
}
According to MSDN this seems to be legal.
For compatibility reasons, SetParent does not modify the WS_CHILD or WS_POPUP window styles of the window whose parent is being changed. Therefore, if hWndNewParent is NULL, you should also clear the WS_CHILD bit and set the WS_POPUP style after calling SetParent. Conversely, if hWndNewParent is not NULL and the window was previously a child of the desktop, you should clear the WS_POPUP style and set the WS_CHILD style before calling SetParent.
We understand that this is a bit erm... 'unorthodox', but the design has been like this for a long time without any other issues.
Interestingly, removing the code that sets the WS_CHILD flag fixes the lost keys problem, but causes a bunch of other issues instead.
According to Raymond Chen
Creating a cross-thread parent/child or owner/owned window relationship implicitly attaches the input queues of the threads which those windows belong to...
We interpret this like AttachThreadInput is called 'under the hood', but do not fully understand what causes keypresses to be lost. Is it some sort of thread-safety issue? A race condition?
By using the AttachThreadInput function, a thread can share its input states (such as keyboard states and the current focus window) with another thread. Keyboard and mouse events received by both threads are processed in the order they were received until the threads are detached by calling AttachThreadInput a second time and specifying FALSE for the fAttach parameter.
We have used Spy++ to monitor the Windows messages that gets sent to the WPF process and it seems that when you type slowly, WM_KEYDOWN is sent together with WM_CHAR. When the input is lost, we still receive the WM_KEYDOWN messages, but no WM_CHAR. This led us to start investigating the .NET framework source. Among other things, we found the following code in HwndSource.cs
switch ((WindowMessage)msgdata.msg.message)
{
case WindowMessage.WM_SYSKEYDOWN:
case WindowMessage.WM_KEYDOWN:
// MITIGATION: HANDLED_KEYDOWN_STILL_GENERATES_CHARS
// In case a nested message pump is used before we return
// from processing this message, we disable processing the
// next WM_CHAR message because if the code pumps messages
// it should really mark the message as handled.
_eatCharMessages = true;
DispatcherOperation restoreCharMessages = Dispatcher.BeginInvoke(DispatcherPriority.Normal, new DispatcherOperationCallback(RestoreCharMessages), null);
// Force the Dispatcher to post a new message to service any
// pending operations, so that the operation we just posted
// is guaranteed to get dispatched after any pending WM_CHAR
// messages are dispatched.
Dispatcher.CriticalRequestProcessing(true);
The variable named _eatCharMessages could perhaps be related, although debugging this code using dnSpy has not made us any wiser.
We're looking for a Win32-wizard to shed some more light on this. Anything that can point us in the right direction would be greatly appreciated. We need to figure out the root cause of the problem in order to decide how to fix it.
Related
I have a Win32 application that determines whether there are any visible, non-iconic, minimizable windows being shown. To the best of my knowledge it's worked fine for Win9x through to Win8.1, but under Windows 10 it often finds several windows that aren't actually visible on the screen.
To try to identify what's going on I've written a simple test application that enumerates and records all such windows. Here's the essence of the EnumWindows callback code:
BOOL CALLBACK EnumFunc( HWND hWnd, LPARAM lParam )
{
if ( IsWindowVisible( hWnd ) )
{
if ( !IsIconic( hWnd ) )
{
const LONG style = GetWindowLong( hWnd, GWL_STYLE );
if ( WS_MINIMIZEBOX & style )
{
// record window info
}
}
}
return TRUE;
}
Most of the phantom windows under Windows 10 belong to background store app processes such as Mail, Calculator, and Photos. These are listed under the Background processes section of Task Manager, and if I use Task Manager to end those background tasks, their phantom window is no longer found by my test application.
In the above screen shot from my test application you can see that all but 1 of the offending windows belong to threads of the same process id 7768, which is ApplicationFrameHost.exe. The final window with process id 11808 is explorer.exe.
I've looked at the phantom windows with Spy++ and can't see any particular style combination that would help in uniquely identifying them.
I've had a suggestion that the undocumented Windows "bands" may be involved, but I've tried using the (undocumented, so this may be wrong) API:
BOOL WINAPI GetWindowBand (HWND hWnd, PDWORD pdwBand);
but it returns a band of 1 for any window, so doesn't differentiate these phantoms.
How to reliably identify these phantom windows?
The approved way of detecting these phantom windows is to use DwmGetWindowAttribute and DWMWA_CLOAKED.
Here's the code I've used:
static bool IsInvisibleWin10BackgroundAppWindow( HWND hWnd )
{
int CloakedVal;
HRESULT hRes = DwmGetWindowAttribute( hWnd, DWMWA_CLOAKED, &CloakedVal, sizeof( CloakedVal ) );
if ( hRes != S_OK )
{
CloakedVal = 0;
}
return CloakedVal ? true : false;
}
Thanks to Scot Br from MS for posting the answer here
Top-level windows of class ApplicationFrameWindow are containers for Windows Store apps. First, here is the window of Mail shown in Spy:
This is truly visible (not phantom). You can tell that it is because the first child is a window of class Windows.UI.Core.CoreWindow. Interestingly, the owner process of the ApplicationFrameWindow is APPLICATIONFRAMEHOST, but the owner process of the Windows.UI.Core.CoreWindow is a different one: HXMAIL. (I've not seen a child window owned by a different process than the parent one before!)
Compare that with a phantom window (as identified in your RWTool):
It is missing the child of class Windows.UI.Core.CoreWindow.
This suggests an answer to your question: If a top-level window is of class ApplicationFrameWindow, iterate it's children. If the first child has class Windows.UI.Core.CoreWindow, the window is visible, otherwise it is not (i.e. it is phantom).
But what if an old-fashioned, non-store app happened to have a top-level window of class ApplicationFrameWindow? It would not have a child of Windows.UI.Core.CoreWindow. Yet it is visible. How to tell this is an ordinary app and not a Windows Store app? I don't have a foolproof way. You could also check for the existence of the other child windows of a Store app: ApplicationFrameTitleBarWindow and ApplicationFrameInputSinkWindow. The chances of a non-Store app having this exact Windows hierarchy is vanishingly small.
EDIT
The ApplicationFrameWindow's (and also Windows.UI.Core.CoreWindow) have the WS_EX_NOREDIRECTIONBITMAP style set:
The window does not render to a redirection surface. This is for windows that do not have visible content or that use mechanisms other than surfaces to provide their visual.
At minimum, you could check for this style instead of special casing ApplicationFrameWindow. Though to see if any content was truly visible, you'd still need to make that depend on whether it has a child of Windows.UI.Core.CoreWindow.
I want to simulate input in games with SendKeys, but I have a hard time.
If I use it with i.e. the letter T, while the cursor in Minecraft is in a textbox (in the main menu), it works, the letter T is written in the textbox.
But with {ESC} it doesn't work. Nothing happens. If I press it manually, it backs to the previous menu. (as it should)
With some applications ESC works:
It works with Discord, Sourcetree, Slack, Chrome, CS2D,
but for some reason it doesn't work with Minecraft, Spelunky, Half-Life.
All of the applications mentioned above were in windowed mode.
Another issue:
If I send 2 to Minecraft while in a text field, it works correctly, 2 is written.
But if I send it while I'm playing, there is no effect. (The character should switch to Item Slot #2)
Same with " " (whitespace). In text fields it works, but the character won't jump in the game.
Code:
[DllImport("USER32.DLL", CharSet = CharSet.Unicode)]
public static extern IntPtr FindWindow(string lpClassName, string lpWindowName);
[DllImport("USER32.DLL")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
public Form1()
{
InitializeComponent();
IntPtr minecraftHandle = FindWindow("GLFW30", "Minecraft* 1.15.2");
if (minecraftHandle == IntPtr.Zero)
{
MessageBox.Show("Minecraft is not running.");
return;
}
SetForegroundWindow(minecraftHandle);
SendKeys.SendWait("{ESC}");
}
I tried it without focus switching: by assigning the SendKey calls to a hotkey, so the target application can be in focus when the SendKeys are called.
The results are the same :\
Do not use SendKeys.Send to messaging between processes working on different runtimes
SendKeys.Send method is from System.Windows.Forms namespace.
This means it is not a Windows Input simulator, but just a little helper for Windows Forms applications. There is no guarantee this method work with another process on different (not .NET) runtime system.
Despite the fact that SendKeys.Send method uses native Windows API, it send key pressing message only of fixed period of time, so game frame handling may not have time to catch this message to manage it. So you may need for separate commands to send message about key down and key up events.
Do not use SendKeys API for messaging with another processes, especially with games.
Also, games can use protection system to rid of automatic bots that can blocks any messages from operation system programming input
So, what you can use?
First, you can try to use PostMessage of user32.dll system library:
const uint WM_KEYDOWN = 0x0100;
const uint WM_KEYUP = 0x0101;
[DllImport("user32.dll")]
static extern bool PostMessage(IntPtr hWnd, uint msg, int wParam, int lParam);
// hWnd - Window handler (can be getted by using GetForegroundWindow/FindWindow methods)
// msg - Key up/down message (WM_KEYUP / WM_KEYDOWN)
// wParam - Virual key code you need to pass to the window
// lParam - Additional parameter for set up key message behaviour.
All virtual key codes can be found on microsoft docs website:
https://learn.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes
Don't forget that you need to wait some time before key releasing. This is needed because games cache inputs between frames, and frame has fixed time to catch input. Just write some delay between key down and up messages.
Also you can set up key message behaviour by lParam. See WM_KEYDOWN and WM_KEYUP parameters. Special thing about WM_KEYDOWN message is if you pressing key on real keyboard long time, operation system repeating WM_KEYDOWN message accordingly. Repeating count can be setted up through lParam. Use it if window you messaging does not react on single keydown message.
PostMessage is low-level system command that can be used for messaging between processes. This command has a low probability to be blocked by protection system (but not zero) and high probability to be received by the game/process you working with. Also it provides opportunity to separate key up and key down messages.
What if PostMessage didn't work?
Try to use hardware scan code instead of virtual key code. Detailed explanation how you can do that described in this answer.
If protection system is really good and PostMessage is blocking even if you use hardware scan code, one thing you can try is to use another keyboard input driver or write it yourself. That driver must replace default system keyboard driver. And you can message it to interact with game. This is the 100% guarantee way to interact with other process through keyboard. But if you use public custom keyboard drivers, there is some probability that protection system blocks it. So you need to write your own driver to message between processes.
I am writing a C# WPF program which sends text messages to another program's window. I have a macro program as part of my keyboard drivers (Logitech g15) which already does this, though it does not send keystrokes directly to the process, but to the currently focused window. It works well but i need to be able to send inputs from my program as well. There are other people using the process so the input text messages from my program needs to be fast enough so that my text does not interfere with their input.
The problem is that when I try to do this with a c# program I get too much delay. The macro program (Logitech G-Series Profiler) sends a command instantly. I have tried the following three commands for sending messages to process. (Listed by order of slowest to fastest)
SetForegroundWindow(_hWnd);
SendKeys.SendWait("{Enter}This is a keystroke input.{Enter}");
It is probably in the name, but this performs the command so slowly that I can actually follow with my eyes the text as it is input letter by letter. I have tried using the “SendKeys.Send” method but I get an error saying: “SendKeys cannot run inside this application because the application is not handling Windows messages.”
PostMessage((IntPtr)_hWnd, (uint)WMessages.WM_KEYUP, (int)key, (int)key);
PostMessage is a bit faster but still not fast enough for the purpose of my program. Besides the method returns before the message has been read by the process, which means two sequential PostMessage calls may not send sequential messages.
SendMessage(_hWnd, 0x100, (int) VKeys.VK_2, (int) VKeys.VK_2);
This is faster than the PostMessage but not nearly as fast as the macro program from Logitech. Also, the receiving program handles the input strangely, apparently not treating it the same way it does "genuine" input from the keyboard.
SetForegroundWindow(_hWnd);
const string text = "This is a keystroke input.";
IInputElement target = Keyboard.FocusedElement;
IInputElement target = InputManager.Current.PrimaryKeyboardDevice.FocusedElement;
var routedEvent = TextCompositionManager.TextInputEvent;
target.RaiseEvent(new TextCompositionEventArgs(InputManager.Current.PrimaryKeyboardDevice, new TextComposition(InputManager.Current, target, text)) { RoutedEvent = routedEvent });
This is the last thing I have tried. It seems instant with the way the text is sent to a process. However, I have only been able to send this to my own program since Keyboard.FocusedElement returns null when I have another program set as foreground window.
If someone can tell me how to get an IInputElement of another window I would sure like to know. Alternatively, if someone has a suggestion for a better method of sending input, I would dearly like to hear it.
Specs: Windows 7, 64bit
Visual Studio 2010, Framework 4
First of all, are you intentionally using WM_KEYDOWN (0x0100) instead of WM_KEYUP (0x0101) in your SendMessage example? This would just press the keys, and never release them, so the application would not process them properly.
Another way worth trying would be to send WM_SETTEXT, assuming the control interprets it correctly (like edit controls or combo boxes).
A last option would be to use SendInput which synthesizes keyboard and mouse input on a very low level, but similarly to you keyboard's macro program, this requires you to activate the correct window and set the focus, which can be quite painful.
Depending on your other's program window type, you could use UI Automation. See this example here:
Add Content to a Text Box Using UI Automation
The requirements I'm up against
About 12 people are using this application, but we only want to allow 4 to close the application through traditional methods (Alt+F4, File > Exit, Close)
If any other method is used (TaskManager, WindowsShutdown) or one of the allowed users close the application, we need to perform some clean up (Closing out some connection channels)
The Code I've used to satisfy said requirements
private void formClosing(object sender, FormClosingEventArgs e)
{
// If a user is allowed to close the application, an empty file (filename)
// will be in the root directory of the application.
if(e.CloseReason == CloseReason.UserClosing && !File.Exists("filename"))
{
e.Cancel = true;
return;
}
// Cleanup
}
The Problem
If a user (not allowed to close) attempts to close the application through traditional methods, then attempts to close using Task Manager the CloseReason enum doesn't seem to reset itself, thus causing Task Manager to pop the prompt to force close, preventing the application from cleaning up.
The Question
Is this a bug, or am I missing something, something that will reset the CloseReason after the FormClosing event has been cancelled.
.NET Reflector is your friend when working out how WinForms is operating.
The Form class has an internal field called closeReason and this is used when generating the event parameter that you examine in the Closing event. This internal field is set in four different places that I can find. These are...
1, The Form.Close() method sets the closeReason = UserClosing.
This makes sense as making a manual call to the Form.Close() method is usually the result of some user action, such as a File->Exit menu option being selected by the user. Clearly this is a user action.
2, The WM_SYSCOMMAND (SC_CLOSE) sets the closeReason = UserClosing.
The WndProc of the Form processes the SC_CLOSE system command by setting the closeReason to UserClosing and the lets the default window proc execute and close the application. This makes sense as this SC_CLOSE is sent when the user presses the window close chrome button or selected the close option from right clicking the title bar. Both are user actions and so setting the closeReason to UserClosing appears correct.
3, WndProc processes message WM_CLOSE (0x10) with closeReason = TaskManagerClosing
WM_CLOSE is sent by task manager and other applications to close a window and if the closeReason is currently equal to None it updates it to TaskManagerClosing. Note this issue with it being updated only if it is None as I think this is a problem for you.
4, WndProc processes messages 0x11 and 0x16 with closeReason = WindowsShutDown
This is not very interesting as you do not care about this scenario but it is just standard processing of shut down messages.
So the core problem you are having is that at no point is the closeReason being reset back to None when you cancel the Closing event. Therefore point number 3 above will never correctly update the value to TaskManagerClosing if that occurs after your cancel. As the closeReasson is an internal field you cannot update it directly. But you can cheat and this is an approach I have used myself in the past. You need to use reflection to get access to the internal field and then reset it to None when you set Cancel=true in your event handler.
I have not tested this code but you need something along the lines of...
PropertyInfo pi = typeof(Form).GetProperty("CloseReason",
BindingFlags.Instance |
BindingFlags.SetProperty |
BindingFlags.NonPublic);
pi.SetValue(this, CloseReason.None, null);
I think you cannot keep your process from shutting down if it's initiated by task manager (that is, OS... he's the 'big boss', it doesn't make sense that you can deny it something like closing your program).
The next best thing is to record the state of the application, and then instantiate another instance of your process with some startup options to take over the state you left. The OS would kill your process, but you will start another one immediately.
Also, if the user clicks in TaskManager "go to process" in the app list, and from there ends process, I don't think you'll be receiving any event at all...
Maybe it would be best if you had a windows service that's running behind the scenes and that keeps track that an instance is running. This way, users probably won't be aware that such process exists since it's not their application, and you can use that keep track of application shutdown.
Kind of a special case problem:
I start a process with System.Diagnostics.Process.Start(..)
The process opens a splash screen -- this splash screen becomes the main window.
The splash screen closes and the 'real' UI is shown. The main window (splash screen) is now invalid.
I still have the Process object, and I can query its handle, module, etc. But the main window handle is now invalid.
I need to get the process's UI (or UI handle) at this point. Assume I cannot change the behavior of the process to make this any easier (or saner).
I have looked around online but I'll admit I didn't look for more than an hour. Seemed like it should be somewhat trivial :-(
If you don't mind using the Windows API, you could use EnumWindowsProc, and check each of the handles that that turns up using GetWindowThreadProcessId (to see that it's in your process), and then maybe IsWindowVisible, GetWindowCaption and GetWindowTextLength to determine which hWnd in your process is the one you want.
Though if you haven't used those functions before that approach will be a real pain, so hopefully there's a simpler way.
#ageektrapped is on the right track, however FindWindow will not search child windows.
For that you will need to use FindWindowEx
Thank you for your answers. Thanks to you here, I figured out how to know if the main window of a process is in front or not:
N.B : of course this needs System.Diagnostic and System.Runtime.Interrop
public bool IsWindowActive(Int32 PID)
{
return IsWindowActive(Process.GetProcessById(PID));
}
[DllImport("user32.dll")]
private static extern
IntPtr GetForegroundWindow();
public bool IsWindowActive(Process proc)
{
proc.Refresh();
return proc.MainWindowHandle.Equals(GetForegroundWindow());
}
You may find that if you call .Refresh() that you get the new top-level window.
If you know the window's title, you can use the Win32 call, FindWindow, through P/Invoke.
You can find the signature here on pinvoke.net
From what I understand MainWindowHandle property of the process you are starting is not valid. If that's the case, you can use FindWindow function (from Win32 SDK) which returns the window handle you need. All you need is the class name of target application's main window. You can obtain it using Spy++ or Winspector. You also need to ensure you have the right window by checking that window's process id using GetWindowThreadProcessId.
At last, I have to say I am not an expert on Win32 and there might be a better solution for your case.
Use Process.GetProcessById(proc.Id); where proc was your splash screen.
Works for me.
Now, how do you get to main window properties in System.Windows.Forms to give it focus w/o using win32?
After all .net is supposed to be a one-stop solution - is it not?
Somewhere in the code, the "real" main window is created. You can just save the window handle at that time and then after the splash screen closes you can set Application.MainWindow to the real window.
The MainWindowHandle property is cached after it is first accessed which is why you don't see it changing even after the handle becomes invalid. GregUzelac's information is correct. Calling Proces.Refresh will causes the next call to Process.MainWindowHandle to re-do the logic to find a new main window handle. Michael's logic also works because the new Process doesn't have a cached version of the MainWindowHandle.