I have been programming against Kinect, and I now want to have games react to what I am doing on the Kinect. It is real easy to send data to notepad for key presses, but much harder to send it to games.
First off, I have been using the WPF Skeleton example from Kinect and building off that for now. I could use the C++ version but my C++ is very rusty, and I would prefer not to.
So here is what I have done so far, I have tried SendKeys, SendInput, keybd_event, Post_Message. None of those make it to games like Burnout Paradise.
I do know GlovePIE input gets to games, but how? Currently my work around/hack, is to use PPJoy, which has sample code in C++ to emulate button presses. I call this via [DllImport] from my WPF app. I then pickup the joystick button presses in GlovePIE and have it convert those to Keyboard Keys. So I go around in a circle, which works but PPJoys driver is not signed, so I can't really share this code as people would have to allow test-signed drivers.
Does anyone know how GlovePIE makes their Keypresses happen? I have posted on the GlovePIE Forums, but no responses. GlovePIE has a little bit of a hack to work with the old openNI kinect drivers, but I am using the standard microsoft version recently released a few weeks ago.
Ok, not sure what the scoop is on answering your own question, but I figured out the whole proper solution and wanted to share as the Checked off Answer.
First Include this using statement
using System.Runtime.InteropServices;
Then Put this in your class this in your class
[DllImport("user32.dll", EntryPoint = "keybd_event", CharSet = CharSet.Auto, ExactSpelling = true)]
public static extern void Keybd_event(byte vk, byte scan, int flags, int extrainfo);
This will import the C method for use in C#.
Now some games / application use Virtual Keys VKey and some (Direct X aka DIKey) use Scancodes. My problem was I used the scan codes improperly so it did not work for some games. I suggest if you do not know what application wants to consume these, you call it twice one with Virtual Key and one for Direct Input Key
Here is an example of two calls for the letter 'A' using Virtual Keys then using Direct Input Keys.
KEYDOWN = 0
KEYUP = 2
//Virtual Key
Keybd_event(65, 0, KEYDOWN, 0);
//Direct Input Key
Keybd_event(0, 30, KEYDOWN, 0);
As you can tell the values for A are different from VK to DIK
Both of the links relate to the HEX values, while my samples above show Base 10 (Integers)
VKeys link http://delphi.about.com/od/objectpascalide/l/blvkc.htm
DIKeys link http://community.bistudio.com/wiki/DIK_KeyCodes
This should also work for SendInput also, but I have not fully verified that.
I don't know GlovePIE, sorry, but perhaps these might help.
PostMessage and SendMessage get different behaviour when dealing with emulating keystrokes.
Would also help to know what message your actually sending, keydown/up, keypress etc.
You may need to do something about changing the focus - maybe the wrong element is selected, or your sending to the wrong (sub)window.
Similarly if you where emulating mouse clicks there can also be checks for where the mouse is to ensure it is still on the clickable area.
Consider also holding a key down triggers a repeat mechanism sending multiple key messages, commonly in games your holding the key down to turn not just tapping it once.
While I will not say this is the best answer, I wanted to round back with the solution I am using. I have found that I can use vJoy http://headsoft.com.au/index.php?category=vjoy which is signed, so it can be used in Windows 7 64 bit. I can the call this extern keybd_event
[DllImport("user32.dll", EntryPoint = "keybd_event", CharSet = CharSet.Auto, ExactSpelling = true)]
public static extern void Keybd_event(byte vk, byte scan, int flags, int extrainfo);
From there I can Keybd_event(CKEY, SCANCODE, KEYDOWN, 0); then X frame later call Keybd_event(CKEY, SCANCODE, KEYUP, 0);
I set vJoy to read in "c" or other keys I send via keybd_event. vJoy reads this properly and then "presses" the associated button which GlovePIE picks up. So in GlovePIE my script look like
A = joystick1.Button1
Z = joystick1.Button2
Which works in the games I have tried.
It is definitely not idea, but it works and allows end user to customize the input via vJoy and GlovePIE.
Related
I'm currently trying to develop an App that enables the user to send Keystrokes to a specific program. I already can send keys and hold down specific keys like that:
[DllImport("user32.dll", SetLastError = true)]
static extern void keybd_event(byte bVk, byte bScan, int dwFlags, int dwExtraInfo);
keybd_event(key, 0, 0, 0); //Start holding the key down
keybd_event(key, 0, KEY_UP_EVENT, 0); //Stop holding the key down
Holding Shift down while sending letters capitalizes them already.
But if I'm holding down a letter (and send it to notepad for example), then just writes that letter once. If I do the same with my physical keyboard, it starts writing the letter until I let go again. Does anyone have a tip or clue how I can manage that?
I already experimented with the stopwatch, but that wasn't working either.
Thanks to #RaymondChen , this question has been answered.
Physical keyboards have a feature called "typematic" which autorepeats characters which are held down. You'll have to emulate that too.
If you want to simulate the repetition of a keystroke, just send multiple KeyDown and KeyUP-Events.
I need to detect the code page the keyboard input is using while user is entering data into application fields.
I tried to use System.Text.Encoding.Default.CodePage; but it gives the code page of what is configured in regional settings.
Then i thought Console.InputEncoding.CodePage; could work, but still, code page is the same as with the above example.
The problem is, user may have a Cyrilic (Windows-1251) code page because of the regional settings, but he may wish to use a different input language. The data user enters is then saved to a file, and that file can be opened in a system which has a different regional settings. Along with text, i am saving code page number, so my app can load the file and display the text correctly. I cannot use Unicode for cross compatibility with a different app which does not support unicode.
Disclaimer: I'm not an expert on the globalization side of the .NET Framework, so it might wrap equivalent functionality somewhere or another. If so, and you can locate it, great—use that instead. I thought maybe Globalization.CultureInfo.CurrentCulture would return the information we're looking for, but alas it does not; instead, it appears to return the system default culture even if the keyboard layout is changed. I stopped investigating. The described approach is guaranteed to work, albeit with a little extra code.
To determine the code page associated with a particular keyboard layout, you can call the Win32 GetLocaleInfo function, specifying the language ID associated with the current keyboard layout. You request this using the LOCALE_IDEFAULTANSICODEPAGE constant.
To call these functions from a .NET application, you will need to use P/Invoke. You'll also need to define function equivalents for some essential macros.
const int LOCALE_IDEFAULTANSICODEPAGE = 0x1004;
const int LOCALE_RETURN_NUMBER = 0x20000000;
const int SORT_DEFAULT = 0x0;
static int LOWORD(IntPtr val)
{
return (unchecked((int)(long)val)) & 0xFFFF;
}
static int MAKELCID(int languageID, int sortID)
{
return ((0xFFFF & languageID) | (((0x000F) & sortID) << 16));
}
static int MAKELANGID(int primaryLang, int subLang)
{
return ((((ushort)(subLang)) << 10) | (ushort)(primaryLang));
}
[DllImport("kernel32.dll", SetLastError = true)]
static extern int GetLocaleInfo(int locale,
int lcType,
out uint lpLCData,
int cchData);
[DllImport("user32.dll")]
static extern IntPtr GetKeyboardLayout(uint idThread);
Use it like this:
// Get the keyboard layout for the current thread.
IntPtr keybdLayout = GetKeyboardLayout(0);
// Extract the language ID from it, contained in its low-order word.
int langID = LOWORD(keybdLayout);
// Call the GetLocaleInfo function to retrieve the default ANSI code page
// associated with that language ID.
uint codePage = 0;
GetLocaleInfo(MAKELCID(langID, SORT_DEFAULT),
LOCALE_IDEFAULTANSICODEPAGE | LOCALE_RETURN_NUMBER,
out codePage,
Marshal.SizeOf(codePage));
When tested with a US English keyboard layout, codePage is 1252. After switching to a Greek keyboard layout, codePage is 1253. Likewise, Turkish returns 1254, and the various cyrillic languages return 1251. Exactly as documented.
It is worth noting that, in the linked documentation, these API functions are indicated as having been superseded. With modern versions of Windows, Microsoft has moved to named locales, first because they were out of room for numeric IDs, and second to enable support for custom locales. But you will need to use the old functions for what you're doing. Modern Windows applications don't use ANSI code pages, either.
However, you do need to be aware of this fact, because it may come back to bite you. There are keyboard layouts that do not have an associated ANSI code page. For these, only Unicode can be used. The above code will return CP_ACP (which is equivalent to the numeric value 0). Handling that is up to you. You will either need to display an error, or save the file as Unicode (albeit breaking the other application, but complying with user expectations).
Finally, I must point out that if you cache the codePage value, it may become stale since the user can change the keyboard layout at any time. It is probably easiest just not to cache the value, determining it each time you perform a save. But if you want to cache it, you will need to handle the WM_INPUTLANGCHANGE message and update your cached value in response.
So, basically what i'm doing is using JNA to set a LowLevelKeyboardProc Keyboard hook, everything works perfectly fine, i can get the values exactly like i want them in java, but the problem i get is when trying to convert to chars, it becomes extremely ennoying handling caps locks, SHIFT keys and tons of other things like everything thats not a-z 0-9 on the keyboard, i was wondering if there is a easier way to do the conversion?
heres the details of what I'm getting from the hook every time a key is pressed
http://msdn.microsoft.com/en-us/library/windows/desktop/ms644967(v=VS.85).aspx
, i Figured it might be best to find a way to manually generate a KeyEvent(Not char, since i need something to handle things like F keys, caps lock button, CTRL button etc etc).
Any help i can get is highly appriciated!.
The Abbot project (http://abbot.sf.net) has a system for mapping keycodes to keychars, using predefined keyboard mappings (it generates a wide variety of keystrokes and records the resulting character output). However, Java does not provide a way for "predicting" the resulting character output given a particular key code.
There may be something within the MS libraries.
I have a string of values, and I want to simulate key press events in a window for each character.
I plan on sending WM_KEYDOWN, WM_CHAR, and WM_KEYUP events to the window (as that is what seems to happen whan a key is manually pressed).
Those messages require an int be sent in the wParam based on a table of virtual key codes. I can loop through the string and get each character, but how do I take that character and convert it to a value that corresponds to the virtual key code? Convert.ToInt32() does not work.
VkKeyScanEx anyone? According to MSDN it:
"Translates a character to the corresponding virtual-key code and shift state."
(You could possibly also use VkKeyScan but beware that it has been superseded by VkKeyScanEx.)
Sending WM_KEYDOWN/UP is troublesome. The application itself already translates the WM_KEYDOWN message into WM_CHAR, using the state of the modifier keys (Shift, Alt, Ctrl) and the keyboard layout. Neither of which you can control, you'll get the wrong character, randomly.
Just send WM_CHAR messages, set the wparam to the character code. No need to worry about lparam, few apps ever use it.
It looks like it takes the ASCII character and turns it into hex. For example, 'A' in hex is 41. According to your chart, A is 0x41, which is right (the 0x detonates hex).
Generally speaking, instead of sending WM_KEYDOWN, WM_CHAR and WM_KEYUP messages directly, you should use SendInput (preferred) or possibly keybd_event (deprecated).
You can use Input Simulator library which provides a high level api for simulating key presses and handles all the low level stuff.
System.Windows.Forms.SendKeys is a WinForms class with static methods for simulating keyboard input on the active window.
The catch is that .NET has no way of focusing windows in another application (the SendKeys docs talk about how to get around that).
The general solution is to use the SendMessage WinAPI function. This link describes SendMessage's signature and provides a sample import.
Oh and, to map VK codes you should use MapVirtualKey - its best to assume the mapping is arbitrary and not logical.
I'm trying to send a key stroke to one application, through PostMessage. I am using too Spy++ to try to understand how to send the message, as I do not fully understand its inner workings.
In this picture, the first item(selected item) was made with an actual key stroke made by myself. The one with a red elipse around it(below) was made with the following code:
WinApi.PostMessage(InsideLobbyHandle, WinApi.WM_KEYDOWN, (int)WinApi.VK_UP, 1);
I guess it must have something to do with the last PostMessage() parameter, but I can't figure out how it really works. I can see in the original key stroke the ScanCode = 48, and in mine its 0, and also fExtended is 1 and in mine is 0. How can I make it look the same?
In http://msdn.microsoft.com/en-us/library/ms646280(VS.85).aspx I cannot understand the last parameter's working.
Simulate keyboard input using SendInput, not PostMessage.
You can't simulate keyboard input with PostMessage.
There are still some caveats with respect to keyboard state/async-state:
The SendInput function does not reset
the keyboard's current state.
Therefore, if the user has any keys
pressed when you call this function,
they might interfere with the events
that this function generates. If you
are concerned about possible
interference, check the keyboard's
state with the GetAsyncKeyState
function and correct as necessary.
The lParam for the WM_KEYDOWN Notification is specified in terms of the bits of the field:
The first 16 bits are the repeat count
The next 8 bits are the scan code
The next bit is 1 for extended key, 0 otherwise
The next 4 bits are reserved and must be 0
The next bit is always 0 (for WM_KEYDOWN)
The next bit is the previous key state
The last bit is always 0 (for WM_KEYDOWN)
A warning: Any solution you build based around PostMessage is going to be very brittle.
Take a look at http://inputsimulator.codeplex.com, it wraps the SendInput method mentioned by Kevin
In Spy++ if you right click on the highlighted (logged message) entry and look at its properties, You can see the exact value of the lParam. You can then use that as your lParam to ensure that the PostMessage leads to similar effects, as the manual action did.