I'm the developer on a game which uses gesture recognition with the HTC Vive roomscale VR headset, and I'm trying to improve the accuracy of our gesture recognition.
(The game, for context: http://store.steampowered.com//app/488760 . It's a game where you cast spells by drawing symbols in the air.)
Currently I'm using the 1 dollar algorithm for 2D gesture recognition, and using an orthographic camera tied to the player's horizontal rotation to flatten the gesture the player draws in space.
However, I'm sure there must be better approaches to the problem!
I have to represent the gestures in 2D in instructions, so ideally I'd like to:
Find the optimal vector on which to flatten the gesture.
Flatten it into 2D space.
Use the best gesture recognition algorithm to recognise what gesture it is.
It would be really good to get close to 100% accuracy under all circumstances. Currently, for example, the game tends to get confused when players try to draw a circle in the heat of battle, and it assumes they're drawing a Z shape instead.
All suggestions welcomed. Thanks in advance.
Believe me or not, but I found this post two months ago and decided to test my VR/AI skills by preparing a Unity package intended for recognising magic gestures in VR. Now, I'm back with a complete VR demo: https://ravingbots.itch.io/vr-magic-gestures-ai
The recognition system tracks a gesture vector and then projects it onto a 2D grid. You can also setup a 3D grid very easily if you want the system to work with 3D shapes, but don't forget to provide a proper training set capturing a large number of shape variations.
Of course, the package is universal, and you can use it for a non-magical application as well. The code is well documented. Online documentation rendered to a PDF has 1000+ pages: https://files.ravingbots.com/docs/vr-magic-gestures-ai/
The package was tested with HTC Vive. Support for Gear VR and other VR devices is progressively added.
Seems to me this plugin called Gesture Recognizer 3.0 could give you a great insight on what step you should take
Gesture Recognizer 3.0
Also, I found this javascript gesture recognition lib in github
Jester
Hope it helps.
Personally I recommend AirSig
It covers more features like authentication using controllers.
The Vive version and Oculus version are free.
"It would be really good to get close to 100% accuracy under all circumstances." My experience is its built-in gestures is over 90% accuracy, signature part is over 96%. Hope it fits your requirement.
Related
I am currently playing around with WPF 3D. I allready got a cube in my application. Now I want to visualize the coordnate axes of the world system. It should look like a coordinate system on paper with lines res. columns as axes like for example the arrows in blender.
Thank you for your help
I can't recommend Helix 3D toolkit enough. It is a comprehensive toolkit for developing 3D WPF applications.
If you are worried that using a toolkit will reduce your discovery of 3D or you will somehow loose control of how you want to do things then I think you will be pleasantly surprised that it actually accelerates your learning and you are in total control.
The current home of Helix3D is on GitHub (also available on NuGet).
Download the source. In the source folder there is HelixToolkit.Wpf.sln which includes a sub project holding all the examples. If you compile the examples sub-project and run it you are presented with >30 really useful examples.
Here is a screenshot of the example browser running that has a demo of what you are asking for; 3D grid with x,y,z vectors at the origin.
I'm currently facing a problem with WPF 3D using C#. To put it simple, I need to animate some simple mechanical part by only moving two of them (one at a time or both together). Here is a simple drawing depicting the situation :
So by moving (translating) vertically P1 or/and P2 parts, the whole thing needs to move accordingly.
I guess it may be possible to do by computing a lot of angles and applying numerous transformations but this is not my goal.
Therefore I would imagine something like attaching the parts together by the means of a pivot point.
What is the preferred way to do this to preview it using WPF 3D?
WPF 3D, Ogre, Mogre, OpenTK... are libraries for display. They have nothing to do with mechanical constraints calculations. But they goes well with physics engines.
WPF 3D is a subset of WPF dedicated to 3D drawing. If you need 2D, then WPF is enough.
As your project looks 2D, you might want to have a look to Farseer Physics which is a port of Box 2D. The feature you need is called joints. Both libraries target 2D games development, but they can be used for simple kinematics animations, and Farseer Physics is doing very well with WPF.
It's a simple problem for any 2D kinematics package.
http://books.google.com/books?id=IGtIWmM2GWIC&pg=PR12&lpg=PR12&dq=c%23+kinematics&source=bl&ots=eCJZLq_i6R&sig=wC42cNOdtw4VX9ElTk4IBDAYtzc&hl=en&sa=X&ei=3YkXU4u1EeHu2wXum4GYDA&ved=0CFsQ6AEwBQ#v=onepage&q=c%23%20kinematics&f=false
I have a set of 3D points mapped onto [0, 1] segments. These points represent simple gestures like circles, waving etc. Now I want to use Hidden Markov Models to recognize my gestures. First step is to extract features for (X, Y, Z) data. I tried to search something useful and found a couple examples: SIFT, SURF, some kind of Fast Fourier Transform etc.
I'm confused which one I should use in my project. I want to recognize gestures using data from Kinect controller, so I don't need to track joints algorithmically.
I had to implement HMM for gesture recognition a year or two ago for a paper on different Machine Learning methods. I came across Accord .NET Framework which helps implement many of those I was looking into, including HMM. It's fairly easy to use and its creator is active on the forums.
To train the HMM I created a Kinect application that would start recording a gesture once a body part was stationary for 3 seconds, it would then record all the points to an output file until said part stopped for 3 seconds again. I then selected the best attempts at the gestures I wanted to train and used them as my training set.
If you are new to Kinect Gesture Recognition and don't need to use HMM I would suggest maybe looking into Template Matching as it's a lot simpler and I found it can be very effective for simple gestures.
I'm working on a similar problem. So far the best material, that I have found is Kinect Toolbox from David Catuhe. Has some basic code for gesture recognition, Kinect data recording and replay.
You can start reading here: http://blogs.msdn.com/b/eternalcoding/archive/2011/07/04/gestures-and-tools-for-kinect.aspx
Have you considered a [trained] Support Vector Machine?
See: LibSVN Library http://www.csie.ntu.edu.tw/~cjlin/libsvm/
The idea would be to define your gesture as a n-dimensional training problem. Then simply train for each gesture (multiple classification SVM). Once trained you map any user gesture as N-dimensional vector and attempt to classify it with the trained model.
I am going to make a game like XNA example game "Platformer1" which comes with the XNA. But I need longer levels which doesn't fit in the screen (like Super Mario levels). How can I manage this kind of level? Do I need to use a 2d camera that follows the sprite? If I do this way how can I load the level? I am a bit confused and I am not sure if I could explain my problem clearly. Hope someone can help?
The tutorial based on Platformer Starter Kit in MSDN has a step Adding a Scrolling Level which guides you through creation of longer levels. The tutorial is very detailed, I highly recommend it.
I couldn't find the tutorial in the section for XNA Game Studio 4.0, but differences should be minimal. According to the comment at the bottom of the page, all you need to change is replace
spriteBatch.Begin(SpriteBlendMode.AlphaBlend, SpriteSortMode.Immediate, SaveStateMode.None, cameraTransform);
with
spriteBatch.Begin(SpriteSortMode.Immediate, BlendState.AlphaBlend, SamplerState.LinearClamp, DepthStencilState.Default, RasterizerState.CullCounterClockwise, null, cameraTransform);
in the tutorial code.
If you want to create a side scrolling game, then I would look into parallax scrolling. A quick google/bing will help you find lots of tutorials. Also, another useful tip is to search YouTube for XNA videos has we a lot of posters share their source code .
Here is a link to Microsofts Parallax Scrolling.
Sounds like you have a few problems ahead of you.
But I need longer levels which doest'n fit in the screen(like super mario levels). How can I manage this kind of levels.
There are several ways to do this, but a fairly easy way would be to have a 2d array (or sparse array, depending on how large your levels are) of a class named Tile that stores info about the tile image, animation, ...whatever.
Yes, you'll probably want a "camera". This can be as simple as only drawing a certain range of that array or a more featured camera that uses transforms to zoom out and translate across your level.
Hopefully this will help get you started.
I've done a decent amount of work in XNA, and from my experience, there are 2 ways to draw a 2D scene:
1) Strictly 2D. This method is much easier, but has a few limitations. There is no "camera" per se, what you do is move everything underneath the fixed 2D "camera". I say "camera" in quotes because the camera is fixed (as far as I know). The upside is that it's easy, the downside is that you can't easily zoom in or out or do other camera effects.
2) 2D in 3D. Set up a 3D world with a 2D plane. This is more flexible, but is also more challenging to work with because you will need to set up a 3D world and 3D camera. If this is your first attempt with making a game, I would highly recommend against this method.
I'm really only familiar with the strictly 2D method, and you would want a list of map objects that have a 2D coordinate. You would also want to store which section of the map you are looking at, I do this with a Rectangle or Vector2 that stores this. This value would move forward as the character moves. You can then take your 2D map objects' coordinate and subtract the (X,Y) of the top-left of what you are looking at to determine an object's screen position. So:
float screenX = myMapObject.X - focusPoint.X;
float screenY = myMapObject.Y - focusPoint.Y;
An other thing to note, use floats or Vector2/3 to store locations, you may not think it's required now, but it will be down the line.
It might be overkill, but my SF project uses XNA to draw a Strictly 2D scene that you can move around: http://sourceforge.net/projects/asteroidoutpost/
I hope this helps.
Have a look at Nick Gravelyns tutorials. They helped me tonne when I was first starting out - Really really worth a look for learning a lot on 2D games.
All the videos are now on youtube here
I'd like to create a game with some kind of a Roulette Wheel with C#.
I have no relevant experience in GUI.
The first thing that comes to my mind when I want to develop such Roulette Wheel, is silver light but I prefer to implement it in a winform (unless you think otherwise).
If you can give me some ideas were to start...
1. How can I create the wheel
2. How can I make the ball go over it (don't want 3D grapics with bouncing and other fantastic ideas)
And please, as simple and basic as possible
Thanks
Asaf
Microsoft released a great framework for 2D and 3D game developemnt - XNA. You can easily add sounds and keyboard events to your game. There is also possibility to put your game onto a windows Forms. Here is a blog post about this by Roy Triesscheijn:
http://royalexander.wordpress.com/2008/10/09/xna-30-and-winforms-the-easy-way/
I've learned XNA 2D basics in one day so you shuld check it out.
Check out also this tutorial, it's really nice and for absolute beginner:
http://creators.xna.com/en-US/education/gettingstarted/bg2d/chapter1
Windows Forms has this amazing feature called nonrectangular forms wherein you can create "shaped" forms like polygons etc. Perhaps that can help you create the wheel.
http://msdn.microsoft.com/en-us/library/6k15y9et.aspx
Child controls placed on that form can act like the wedges on the wedges. Again, these "child controls" themselves could be nonrectangular forms.
I think it would be way easier if you opt for WPF for this instead of Winforms. WPF offers a wide range of controls and features that seem tailormade for such applications.
You can create a gif of a spinning wheel the slowly slows down. Then you can plot the known relative co-ordinates of each of the balls finishing positions.
For example 00 might be at the top when the animation has finished. In your page generate a random number (pseduo random is fine for a roulette wheel where no real money is being staked!), and have a ball gif. Use javascript to 'spin' the ball in the opposite direction of the wheel gif and slowly come to a stop the same time as the wheel animation, finishing at your plotted known co-ordinates.
Hope this gives you some ideas. There are several ways to do this!