I am developing a C# application that gives the user the possibility to set some specific keystrokes that the applications will reproduce when it will be launched in execution. As far as I was testing my application in notepads or other simple programs it worked just using SendInput or InputSimulator, but as soon as I tried it with a video-game the emulated input could not be received by the game.
The strange thing is that actually the input is received by the game but only if I am using the game chat.
To be clearer I will make an example:
I set in my application to reproduce the key w.
I launch my application and then I launch a FPS game like CounterStrike.
When the applications emulates the pression of the w key my character in the game doesn't move! But if i click on the chat and try to write something in the chatbox the emulated input is recognized and I can see the "w"s being written in the chatbox.
Game engines usually don't get their input from the usual Windows API; since this one lets you read only one character at once. In games, however, you can press several keys at once. For instance you might press wa together in order to move your character in forward-left direction. Games check the keyboard state at each game loop instead of reading the characters which have been typed. The character “w” might have been typed only once, but the w key might have been in pressed-state during 8 loops. Game engines seem to access the keyboard through a low-level function. This is why commands like SendKeys injecting their keystrokes at a higher level have no effect.
Related
Problem:
So I have a physical device with buttons and joysticks on it, and I'm using it to move the mouse around and click on icons, etc.
To do this I have a C# program that reads a COM port and uses InputSimulator to fake keyboard and mouse actions. (Btw this app is a graphical application with its own window, since it also displays debug data on the screen.)
This works fine in Windows, in that I can use it to navigate around the OS, open programs, etc. I'm outputting a variable "maxTimeBetweenReadings" and it doesn't really go above 20 or 30ms.
However, I then open Steam and run a computer game. My device works fine at first and I can move around the start menu. But on entering a level, comms from the device becomes incredibly choppy and the game is unplayable. My maxTimeBetweenReadings value shoots up to over a second.
Things I have tried:
I have tried using the real mouse and keyboard, which is still perfectly smooth, so I know it's not that the game is slowing down my whole computer.
I have tried setting the priority of my C# program to Realtime (though I don't want to do this in production) and the priority of the Thread it spawns to Highest. This makes no difference that I can see.
I have commented out all other activity (writing/reading files, etc) in the C# program, even though I don't think that should make a difference at all. Again no difference, but then I wasn't hopeful.
My understanding is that Windows has claimed the COM port for my C# program and now won't let anything else talk to it, so even if the game wants to interfere with it for some reason, it shouldn't be able to?
And even if the game was somehow able to mess around and interrupt the comms, wouldn't the fact that the C# program is running at Realtime priority mean that the game should never be given a chance to do so?
However there are obviously gaps in my understanding, because it doesn't line up with what I'm seeing.
We need to be able to give away this device with the reassurance that it can work with ANY game, no matter how said game was implemented, no matter what awkward things it tries to do.
If I have to go game by game to make sure they aren't interfering somehow (and I still don't understand how they can) then I'll be here for a thousand years.
I am trying to send keyboard inputs to Minecraft to move the player, however when I try using SendKeys.SendWait("W"); nothing happens. If I open the chat in Minecraft it types "W" in chat, however outside of chat my inputs seem to be ignored. Thanks.
Edit: I have tried using SendInput as well as InputSimulator both having the same effect.
Basically Windows has three protection ring. By doing SendKeys you are sending a ring 3 command to the application. However DirectX only listens to ring 0 and ring 1 (possibly ring 2) commands to reduce the fraction delay caused by command passing through a driver to application.
So in order to make DirectX games react to the event you sent you must send it at driver level. You can simulate a ring 2 driver input by pinvoke WINDOWS api SendInput with scan code (don't use virtual code).
If scan code doesn't work then the game might be blocking ring 2 commands for anti-hacking purpose. In that case you would need to write a driver + a virtual hardware to send ring 1 commands directly. (do not try this if you are not experienced. a Blue screen of death or even corrupted system may result if a mistake is made)
I solved it by using InputSimulatorPlus
https://github.com/TChatzigiannakis/InputSimulatorPlus
InputSimulator s = new InputSimulator();
s.Keyboard.KeyDown(VirtualKeyCode.VK_W);
this just runs forward, to stop use s.Keyboard.KeyUp(VirtualKeyCode.VK_W);
you can also use s.Keyboard.KeyPress(VirtualKeyCode.VK_W); and this will just click the "W" key.
So in this maze kind of game I'm making, I will show the maze to the player for 30 seconds.
What I don't want is that the player taking screenshot of the maze.
I want to do something like Snapchat, or Instagram, how it detects when you take a screenshot of a snap/story.
I'm using C#. It can also prevent user to take screenshot. I don't mind.
Is there a possible way to detect when the user takes screenshots or prevent it in Unity?
No, you can't detect this reliably. They also could make a photo with a digi cam. Furthermore there are endless ways to create a screenshot and the os has no "callback" to inform an application about that. You could try to detect the "print screen" key but as I said there are other screenshot / screen recording tools which could use any hotkey or no hotkey at all. I have never used Snapchat but it seems it's not safe either.
There are even monitors and video projectors which have a freeze mode to keep the current image. You could also run your browser in a virtual machine. There you can actually freeze the whole virtual PC or take screen shots from the virtual screen and an application running inside the VM has no way to even detect or prevent that.
I once had to do something similar. If you just want to do what snapchat did then it can be done but remember that as long as the app is running on anyone's device instead of your server, it can be de-compiled, modified and compiled again so this screenshot detection can be circumvented.
First of all you need to know this about Apple's rule:
2.5.9 Apps that alter or disable the functions of standard switches, such as the Volume Up/Down and Ring/Silent switches, or other native
user interface elements or behaviors will be rejected.
So, the idea of altering what happens when you take a screenshot is eliminated.
What you do is start the game, do the following when you are showing the show the maze to the player for 30 seconds:
On iOS:
Continuously check if the player presses the power and the home button at the-same time. If this happens, restart the game and show the maze to the player for 30 seconds again. Do it over and over again until player stops doing it. You can even disconnect or ban the player if you detect power + the home button press.
On Android:
Continuously check if the player presses the the power and volume down buttons at the-same time. Perform the-same action described above.
You cannot just do this with C#. You have to use make plugins for both iOS and Android devices. The plugin should use Java to the the detection on android and Object-C to do the detection for iOS. This is because the API required is not available in C#. You can then call the Java and Objective-C functions from C#.
Other improvement to make:
Check or external display devices and disable them when you are
showing the maze to the player for 30 seconds. Enable them back
during this time.
When you detect the screenshot button press as described above,
immediate take your own screenshot too. Loop through images on the player's picture gallery and load all the images taken that day.
Compare it with the screenshot you just took and see if they match.
If they do, you are now very sure that the player is trying to cheat.
Take action like banning the player, restarting the game or even
trolling the player by sending their screenshot to the other player. You can also use it as a proof to show that the user is cheating when they complain after being banned.
Finally, you can even go deeper by using OpenCV. When you are
showing the player the maze for 30 seconds, start the front camera of
the device and use OpenCV to continuously check if any object other
than the player's head is in front of the camera. If so, then the
player is trying to take a screenshot with another device. Take
action immediately. You can use machine language to train this.
How far to go depends on how much time you want to spend and how much you care about player's cheating online. The only thing to worry about is players de-compiling the game and removing those features but it is worth implementing.
My Android phone takes screenshots differently. I swipe down from the
top of the screen and select the "Capture" option.
Nothing is always the-same on Android. This is different on some older or different Android devices. You can detect swipe patterns on the screen. The best way to do this is to build a profile that handles each Android device from different manufactures.
For those commenting, this is possible to do. You must do it especially if it is a multiplayer game. Just because a game can be hacked does not mean that a programmer should not implement basic hack prevention mechanism. Basic hack prevention mechanism should be implemented then improved as you get feedback from players.
So I created an UWP App that can record several Audio Lines and save the recordings to MP3 files for in-game multi-line recording that I can later edit separately (game audio, microphone, game comms, voice comms) as NVidia ShadowPlay/Share does not support this yet. I achieve this multi-line setup with VAC.
I have a version of this tool written in regular Windows WPF C# and I have a system-wide HotKey Ctrl+Alt+R that starts/stops recording so when I'm in a full screen game, I can start/stop recording without exiting full screen mode (switching window focus).
Can a global (system wide, app window not in focus) HotKey that triggers some in-App event be achieved in a UWP App? I know the functionality is not supported for other platforms. But I only need it to run on Windows 10 Desktop and the HotKey support is mandatory. Or can I achieve my goal in any other way for UWP Apps?
GOAL: System wide key combination to trigger in UWP app event without switching Window focus and messing with full-screen games.
at the moment it is not possible to solve this task thoroughly.
You are facing two limitations of UWP and can be only partially solved:
Lifecycle: UWP apps go in suspended state when they are not focused. They just "block" to consume less resources (and battery). This is a great feature for mobile devices, but is bad news for you project. You can solve this by requesting "ExtendedExecutionSession" which will guarantee that your app never falls asleep when out of focus if "attached to wallpower".
Detect input without focus. It's clearly stated on MSDN that UWP doesn't support keyboard HOOKS (this refers to SetWindowsHookEx). They reinvented "GetAsyncKeyState", now it works only when the Windows is focused. Indeed you can find that under CoreWindow.GetAsyncKeyState().
If you only need to use F Keys as hotkeys you can still do something, like "press F2 when the app is minimzed to activate a function".
Use Stefan Wick example. He solved part of the problem.
Instead if you need to listen to lots of keys (or mouse events) there isn't a way. You can't right now.
Curiosity
UWP has restricted capabilities, one of which called "InputObservation".
At the moment it is not documented and impossible to implement (unless you are a select Microsoft Partner), but it should allow apps to access system input (keyboard/mouse..) without any limitation and regardless its final destination.
I think this feature is the key for system-wide inputs detection.
I am not able to find a way to implement it.
Kind Regards
so I have received an external motion controller device (Myo) and I wish to create an application where certain motions will basically simulate a keystroke or keypress globally (doesn't matter about what application). This will happen while my program is running in the background so it can receive motion inputs and output as a keyboard press.
An example would be if I were to be playing Baseball game in the foreground (also full screen) and I do a pitching motion, the program will output the key which will do a pitch in game (whichever key it might be).
I have looked into the SendKeys class in C# but I feel there might be limitations as to what it can do (specifically global keypress sending).
Is there a good way where I can possibly write a program so I can map the actions with my motion controller to a keypress using C#? It would also be good if it can do key_down and key_up for key holdings.
The most direct way to accomplish truly global key-presses is to emulate a keyboard. This will involve creating a keyboard driver that somehow provides access to your background program. However this involves kernel programming which is quite complex.
An alternative is to use the SendKeys API combined with some logic to find the currently active application.
I know this isn't a C# solution, but the Myo Script interface in Myo Connect was essentially built for this purpose and would probably be the easiest way of testing things out if nothing else.
To send a keyboard command using Myo Script you can use myo.keyboard() (docs here).
If you want the script to be active at all times, you will need to consistently return true in onForegroundWindowChange() and pay attention to the script's location in the application manager. Scripts at the top of the application manager will be checked first, so your script may lose out if there is another one above it that 'wants' control of a given application.