I am wondering if it is possible to implement an interface similar to Apple's tvOS in WPF with Kinect v2 whereby the hovering of a hand on the screen would focus on the closest button in a grid layout filled with buttons as illustrated in the link below:
https://developer.apple.com/tvos/human-interface-guidelines/images/overview-clear.mp4
https://codepen.io/anon/pen/KvKOzo Mockup illustrating intended interface
If it is possible to implement such an interface, can somebody provide any guidelines, pseudocode or libraries that can help achieve such an interface using WPF and Kinect v2.
You'll need to transform your Skeleton space to a 2d space (color space for instance) and map that space to your WPF view, you'll then have a stream of x and y coordinates of your hand.
Then you can do a AABB collision detection to determine if your hand "touches" the button.
There's some Kinect example projects on:
http://kinectforwindows.codeplex.com/
Controls-Basic-WPF seems like a good starting point.
Related
Since I don't have 10 reputation to post an image, I am going to leave a link of the game picture. http://thinkfun.com/mathcounts/play-rush-hour
This is what I want to create. A game where a rectangle must pass through other rectangles through the hole to win the game. The main rectangle can only move forward and other rectangles can move either right, left, up or down depending on their position. You must move the other rectangles in a way that they could make a way for the main rectangle to pass.
Here is my plan how I am going to create this game:
Create all the forms, import all the images and graphics to get ready to work with them.
Write code for 1 rectangle that would be able to move around.
Write an algorithm that wouldn't allow for rectangles to go through each other.
Make a winning hole.
Code other rectangles, make a full game.
Make fancy animations.
Import statistics : score, move counter and etc.
Here are my questions that I wanted to ask before trying to develop this game.
(Oh, and by the way, I am using Visual Studio C# form )
I assume that these rectangles are not going to be buttons, then what kind of a "button" (or whatever you call those extra features ) should I use?
How do I make a skin for those rectangles? So I can make it look like wood and make a nice skin for the main rectangle.
These are all my questions so far. I would like to know how to overcome those problems and I would like to hear some extra tips. Thank you very much for your time!
Use Graphics to draw your rectangles. Else make an object to hold info for it and you can import an Image as your "skin".
See number 1.
First off: Do you want to create a game or learn winForms? If the answer is create a game. Chose a Game Engine (Unity is nice, and uses c#) and work with that instead.
If you want to learn winForms you'd probably want a PictureBox, then add your code your code in OnMouseUp, OnMouseDown, etc. Good luck :)
I am trying to write a Tetris game using Kinect. There are two game players. One of the game players will move the blocks and the other will define the shape of the blocks using body positions.
Now, the challenge I am facing is : How do I map the body position to a block?
For example I have this shape given below.
I want to identify it as a 'T' shape block. How do i Do it?\
Here is what I thot:
I will store sample images of all the blocks (total 10) and compare the skeleton image from camera to these stored images and then display the block to which the camera has a match.
However, this is the toughest part. How do I find a match? Even if I have a T shaped sample image, what processing do I do with this skeleton so that It resembles a T shape before the comparison can be done?
I would suggest using a gesture recognition library. Such as:
https://github.com/EvilClosetMonkey/Fizbin.Kinect.Gestures
http://kinecttoolbox.codeplex.com/
You can define the gestures as a static held position -- for example, "Menu" in the Fizbin.Kinect.Gestures library. When the particular gesture is recognized an event is fired and you can then act upon it, by showing the shape you desire.
I'm attempting to adapt a MSSurface application to allow use of a Kinect. Using the code4fun libraries, I'm able to generate an event from the Kinect when a user puts their hand towards the screen, but what I'm missing is how to trigger a ScatterViewItem's touch or click event to grab item, and then release it once finished moving. from the kinect skeleton model I can get adjusted x/y co-ordinates which i could apply if I can trap the right events in the ScatterViewItem.. And code suggestions would be appreciated...
regards,
Rob
If you are just looking to move the item, the easiest thing is to set the ScatterViewItem's Center property to the translated x/y coordinates. You can then control when the item is 'grabbed' fairly easily using whatever conditions you want.
If you are also after pinch/zoom, you'll have to do some playing around. Since the Kinect doesn't have the resolution to detect the fingers pinching and zooming, you could implement this by mapping the Z coordinate of the hand to preset sizes on the grabbed ScatterViewItem.
I am making a WPF program with the possibility to modify data graphically in 3D. In order to give the user the option to select multiple graphical objects at the same time, I want to implement a selection rectangle. (Just like the one in windows explorer.) A common functionality in programs like this one is to have 2 different functions for the selection rectangle, and that the user can somehow choose which of the methods should be used.
Any object that is partially or completely inside the rectangle is selected.
Only objects that are completely inside the rectangle are selected.
The 2nd method is straight forward by using the bounding box of each object, and check if it is inside the rectangle. The 1st one on the other hand, seems to be quite some work. All my graphical objects are complicated 3D figures, and can be rotated by the user in any way. At the moment I am unable to find any other way than checking if any of the triangles in the mesh of any of the objects cross my 2D rectangle, and that can be quite time consuming.
I have little experience with WPF 3D, but I have done this before in OpenGL. Then I could tell OpenGL to draw a specific area of the screen, and the collect a list of objects that was visible in the specific area. All I needed to get the functionality I wanted was about 5 lines of code.
I guess my question is this:
Is there a way to do this with WPF 3D, similar to the OpenGL approach?
If not, is there any other smart way to find all objects (Visual3D) in a viewport that is partially behind a 2D rectangle?
I refuse to believe I am the only one with this kind of problem, so I hope a clever mind can point me in the right direction.
Regards,
Sverre
Thank you for your answer!
The 2D-rectangle is just in front of the camera and extending infinitely forward. I want to get any object that is partially or completely inside that frustum.
The camera we are using is an orthographic or perspective projection camera (System.Windows.Media.Media3D.ProjectionCamera). The reason we are not using the matrix camera is that we are using a 3rd party tool that does not support the matrix camera. But I am sure there is a way to get the matrix from a projection camera as well, so that is hopefully not the problem.
In theory your solution sounds like just what we need, but I am not sure how to proceed. Do you have any links to sample-code, or can you give some more hints on how to actually implement this?
Btw: Since we are working with WPF, we do not have direct access to DirectX. At least that’s what we have concluded after some research. You mention use of the z-buffer, which we haven’t been able to access through WPF. If you know a way to access the z-buffer, it’s greatly appreciated! This is of-topic, but we have struggled to disable the z-buffer for some time, but have given up…
Best regards,
Sverre
Is your intersection region a 2d rectangle or a frustrum based at a 2d rectangle and extending infinitely forward (or perhaps to some clipping limit)? If it can be construed as a viewing frustrum, then you can leverage the existing capabilities of the graphics system to render the scene using a Camera View and Projection that corresponds to your originating rectangle, with all lighting and shading disabled and colors chosen specifically to 'tag' the different objects in your scene. This means you can use the graphics hardware to perform the clipping/projection as a 'rendering' operation, then simply enumerate the pixel values as 'tags' to determine the objects present in the rectangular view.
If you need to restrict selection to an actual 2d slice (or a very shallow frustrum), you can use the Z-buffer (if you can get access to it) to exclude tagged pixels that are outside the Z range of your desired selection frustrum.
The nice thing about this approach is that you probably already have the Camera matrix (it's the same matrix used for your window for selection) and only need to change the Projection matrix to be a sub-set of the viewing window.
A 'smart' way would be to transform the rectangle into a box using the Camera's matrix
And then do a intersection of all the objects and the box.
HI all,
I'm developing a software to control light show ( through DMX protocol ), i use C# and wpf to develop my main software (.net 4.0)
To help people preview their show, i would like to make a live 3D visualizer...
First, i thought that i could use wpf 3D to make the visualizer, but i need to work with light ..
My main application should send property ( beam angle, orientation (X,Y), position(X,Y), Brush ( color,shape,effect)) to the 3D visualizer
But i would like to be able to move light (position in the scene) by mouse during execution and had value in return...
So ..
Does XNA is the easiest way to doing that ?
Can you help me for that :
Generating light (orientation , bitmap like filter in front of light )
Dynamically moving object with mouse and get position in return
Dynamically add or remove fixture
All of your advice, sample, example are very welcome ... I don't espect to have a perfect result at the first time but i need to understand the main concepts for doing that
Thank You !!
XNA does not contain any functionality for managing a "scene" - you will have to implement that yourself. For example: you might make a Light class containing the information about your light (position, orientation, etc), and then have a List<Light> of them, which you update and render yourself.
I will now assume that you have a 3D model of a "Light" (as in: the metal box containing the lightbulb) and also a 3D model of a stage. And that you can figure out how to render them - there are plenty of tutorials online for simple model rendering in XNA. Here is a starting point.
So your 3rd requirement ("Dynamically add or remove fixture") should be fairly simple once you can render things. Just add and remove Lights from your List of them based on user input. See the Input namespace.
And your 2nd requirement ("Dynamically moving object with mouse and get position in return") should also be simple. If you want your user to move lights by clicking and dragging, just keep track of the mouse position between frames and apply that as an adjustment to the clicked Light's position (or rotation).
To figure out which Light the user clicks in the first place, a good starting point is the Picking Sample.
I am assuming here that the user will click the Light (metal box) itself to move/rotate it. If you would rather have the user click and drag the endpoint of the light (the spot it projects) - that is more difficult. One idea that comes to mind: Intersect a ray from your Light with the stage to find the centre of the projected spot. At that point draw a dummy "handle" object (like a sphere) that the user can click and drag around. When the user finishes dragging, figure out the new orientation for the Light to make that the new centre.
Finally your 1st requirement ("Generating light (orientation , bitmap like filter in front of light )") is the tricky one. My understanding of this is that you want a way to draw the endpoint of the beam of light on your stage model? If so, what you are looking for is called Projective Texture Mapping. Presumably you will have a circular texture for basic lights, and perhaps other textures for gobos.
The quick-and-dirty way to do this would be to draw your stage model, once for each light, with additive blending (so that each light added with the other lights and the black had no effect), with a colour set to whatever you want the light colour to be, and with a black and white texture (a white circle on black background) drawn with TextureAddressMode.Clamp, with a shader that draws the texture with projective texture mapping which is set up with the light as the projection point.