Develop Reference

about wiimote wrapper class

I'm interested in how this is working and looked at the code. The important part is the warp matrix construction done using the computeSquareToQuad and computeQuadToSquare functions, but I don't understand them. Can you do an explanation or give some references about that?

These two methods are used for translating camera space coordination and display coordinates to each other (computeSquareToQuad for translating from camera coordinates to display and computeQuadToSquare for reverse of it),
When you look at the world through a camera, the result is a flat image and everything is distorted according to perspective rules. (for example squares transform into trapezoids). this distortion can be encapsulated by a warping matrix called a planar homography.
you essentially need a 3x3 matrix for calculation (note the normally 4x4 matrix is used because it can be easily integrated in 3D pipelines)
for more information have a look at
http://www.cs.utoronto.ca/~strider/vis-notes/tutHomography04.pdf
http://www.youtube.com/watch?v=fVJeJMWZcq8

matching Kinect's Skeleton data to a .fbx model in XNA

My question is about a school project that I'm working on.
It involves mapping 3D models of clothing (like a pair of jeans) on a skeleton
that is generated by my Kinect camera.
An example can be found here: http://www.youtube.com/watch?v=p70eDrvy-uM.
I have searched on this subject and found some related threads like:
http://forums.create.msdn.com/forums/t/93396.aspx - this question demonstrates a way using brekel for motion capturing. However, I have to present it in XNA.
I believe that the answer lies in the skeleton data of the 3D model (properly exported as a .FBX file). Is there a way to align or match that skeleton with a skeleton that the Kinect camera generates?
Thanks in advance!
Edit: I am making some progress. I have been playing around with some different models, trying to move some bones upward (very simple use of CreateTranslation with a float that is calculated on the elapsed game time), and it works if I choose the rootbone, but it doesn't work on some bones (like a hand or an arm for example). If I track al the Transform properties of that bone including the X, Y, and Z properties then I can see that something is moving.. However the chosen bones stays in it's place. Anyone have any thoughts perhaps..?

If you are interested, then you'll find a demo here. It has source code for using Real-time Motion capture using the Kinect and XNA.

I have been working on this off and on for a while now. A simple solution I'm using right now is you use the points the nui skeleton tracks to match to the rotations of various joints in a .fbx model. The fbx model will most likely have many more joints then what are tracked and for those you can just iterate a rotation.
The fun part:
The Kinect tracks skeleton joint position in skeleton space -1 to 1 while your models need rotations in model space. Both of them provide child position or rotation in relation with their parent bone in the hierarchy. Also the rotations you need for a .fbx model are around an arbitrary axis.
What you will need is the change from the .fbx model in its bind pose to the pose represented by the kinect data. To get this you can do some vector math to find the rotation of a parent joint around an arbitrary axis and rotate it accordingly then move on down the skeleton chain.
Say you have a shoulder we will call point a and the elbow we can call point b on the bind pose of the fbx model. We will have a point a' and b' on the skeleton.
So for the bind model we will have a vector V from the shoulder to the elbow.
V = b - a
The skeleton model will also have a vector V'
V' = b' - a'
So the axis of rotation for the shoulder will be
Vhat = V cross-product V'
The angle of rotation for the shoulder around Vhat
Theta = ((V dot-product V') / magnitude(V) ) * magnitude(V')
To you will want to rotate the shoulder joint on the fbx model by theta around Vhat.
The model may seem to flop a bit so you may have to use some constraints or use quaternions or other things that help it not look like a dead fish.
Also I'm not sure if XNA has a built in function to rotate around an arbitrary axis. And if someone could double check my math I'd appreciate it, I'm a little rusty.

The Kinect SDK delivers only points of body parts like head postion or right hand position. Seperately you can also read the depth stream from the Kinect sensor.
But currently the Kinect doesn't generate a 3D Model of the body. You would have to do that by yourself.

I eventually settled for the Digital Runes option (with Kinect demo), which was almost perfect apart from a few problems that I wasn't able to solve.
But because I had to hurry for school, we decided to turn the project around completely and our new solution did not involve the Kinect. The Digital Runes examples did help a lot though.

How do you transform a Skinned Mesh during mesh loading and processing?

I created my own skinned mesh loader. It's working fine, but my problem is I don't know how to transform (scale & rotate) the skinned mesh so that the transformations are "baked" onto the vertices. If it were just a geometry, transforming the vertices are a piece of cake, but now that skinning info is involved, if I do a scale for example, my mesh gets all stretched. I know I need to transform my skinning data too, but which parts? All the Bind Pose matrices? The Inverse Bind Pose Matrices? I can't seem to understand how to go about this.
My implementation is in C# & OpenTK and I am specifically loading Skinned Collada files exported from Blender 2.6.
Thanks in advance.

I don't know C# and OpenTK, but I try to help on the theoretical side. Vertices are transformed by weighted global transform matrix. To form a global transform, you need to concatenate local transform of each joints. To create a local transform, you need to concatenate the local translate, rotate and scale. The weight would come from the joint. So I think you need to get joint local rotation/translation/scaling of your bind pose, then manipulate those local matrix and form them to global matrix. After that, you apply weights to the global transformation then transform the vertices.
The following link may be similar to your question.
COLLADA: Inverse bind pose in the wrong space?
I created this collada file player, but use C++.
http://www.youtube.com/watch?v=bXBfVl-msYw

WPF: Finding 3D-visuals that are partially inside a 2D rectangle

I am making a WPF program with the possibility to modify data graphically in 3D. In order to give the user the option to select multiple graphical objects at the same time, I want to implement a selection rectangle. (Just like the one in windows explorer.) A common functionality in programs like this one is to have 2 different functions for the selection rectangle, and that the user can somehow choose which of the methods should be used.
Any object that is partially or completely inside the rectangle is selected.
Only objects that are completely inside the rectangle are selected.
The 2nd method is straight forward by using the bounding box of each object, and check if it is inside the rectangle. The 1st one on the other hand, seems to be quite some work. All my graphical objects are complicated 3D figures, and can be rotated by the user in any way. At the moment I am unable to find any other way than checking if any of the triangles in the mesh of any of the objects cross my 2D rectangle, and that can be quite time consuming.
I have little experience with WPF 3D, but I have done this before in OpenGL. Then I could tell OpenGL to draw a specific area of the screen, and the collect a list of objects that was visible in the specific area. All I needed to get the functionality I wanted was about 5 lines of code.
I guess my question is this:
Is there a way to do this with WPF 3D, similar to the OpenGL approach?
If not, is there any other smart way to find all objects (Visual3D) in a viewport that is partially behind a 2D rectangle?
I refuse to believe I am the only one with this kind of problem, so I hope a clever mind can point me in the right direction.
Regards,
Sverre
Thank you for your answer!
The 2D-rectangle is just in front of the camera and extending infinitely forward. I want to get any object that is partially or completely inside that frustum.
The camera we are using is an orthographic or perspective projection camera (System.Windows.Media.Media3D.ProjectionCamera). The reason we are not using the matrix camera is that we are using a 3rd party tool that does not support the matrix camera. But I am sure there is a way to get the matrix from a projection camera as well, so that is hopefully not the problem.
In theory your solution sounds like just what we need, but I am not sure how to proceed. Do you have any links to sample-code, or can you give some more hints on how to actually implement this?
Btw: Since we are working with WPF, we do not have direct access to DirectX. At least that’s what we have concluded after some research. You mention use of the z-buffer, which we haven’t been able to access through WPF. If you know a way to access the z-buffer, it’s greatly appreciated! This is of-topic, but we have struggled to disable the z-buffer for some time, but have given up…
Best regards,
Sverre

Is your intersection region a 2d rectangle or a frustrum based at a 2d rectangle and extending infinitely forward (or perhaps to some clipping limit)? If it can be construed as a viewing frustrum, then you can leverage the existing capabilities of the graphics system to render the scene using a Camera View and Projection that corresponds to your originating rectangle, with all lighting and shading disabled and colors chosen specifically to 'tag' the different objects in your scene. This means you can use the graphics hardware to perform the clipping/projection as a 'rendering' operation, then simply enumerate the pixel values as 'tags' to determine the objects present in the rectangular view.
If you need to restrict selection to an actual 2d slice (or a very shallow frustrum), you can use the Z-buffer (if you can get access to it) to exclude tagged pixels that are outside the Z range of your desired selection frustrum.
The nice thing about this approach is that you probably already have the Camera matrix (it's the same matrix used for your window for selection) and only need to change the Projection matrix to be a sub-set of the viewing window.

A 'smart' way would be to transform the rectangle into a box using the Camera's matrix
And then do a intersection of all the objects and the box.

.NET high level graphics library

I am programming various simulation tools in C#/.NET
What I am looking for is a high level visualization library; create a scene, with a camera with some standard controls, and render a few hunderd thousand spheres to it, or some wireframes. That kind of thing. If it takes more than one line to initialize a context, it deviates from my ideal.
Ive looked at slimDX, but its way lower level than im looking for (at least the documented parts, but I dont really care for any other). WPF perspective looked cool, but it seems targeted at static XAML defined scenes, and that doesnt really suit me either.
Basically, im looking for the kind of features languages like blitzbasic used to provide. Does that exist at all?

I'm also interested in this (as I'm also developing simulation tools) and ended up hacking together some stuff in XNA. It's definitely a lot more work than you've described, however. Note that anything you can do in WPF via XAML can also be done via code, as XAML is merely a representation of an object hierarchy and its relationships. I think that may be your best bet, though I don't have any metrics on what kind of performance you could expect with a few hundred thousand spheres (you're absolutely going to need some culling in that case and the culling itself may be expensive if you don't use optimizations like grid partitioning.)
EDIT: If you really need to support 100K entities and they can all be rendered as spheres, I would recommend that you bypass the 3d engine entirely and only use XNA for math. I would imagine an approach like the following:
Use XNA to set up Camera (View) and Perspective matrices. It has some handy Matrix static functions that make this easy.
Compute the Projection matrix and project all of your 'sphere' origin points to the viewing frustrum. This will give you X,Y screen coordinates and Z depth in the frustrum. You can either express this as 100K individual matrix multiplications or multiplication of the Projection matrix by a single 3 x 100K element matrix. In the former case, this is a great candidate for parallelism using the new .NET 4 Parallel functionality.
If you find that the 100K matrix multplications are a problem, you can reduce this significantly by performing culling of points before transformation if you know that only a small subset of them will be visible at a given time. For instance, you can invert the Projection matrix to find the bounds of your frustrum in your original space and create an axis-aligned bounding box for the frustrum. You can then exclude all points outside this box (simple comparison tests in X, Y and Z.) You only need to recompute this bounding box when the Projection matrix changes, so if it changes infrequently, this can be a reasonable optimization.
Once you have your transformed points, clip any outside the frustum (Z < 0, Z > maxDist, X<0, Y<0, X>width, Y>height). You can now render each point by drawing a filled circle, with its radius proportional to Z (Z=0 would have largest radius and Z=maxDist would probably fade to a single point.) If you want to provide a sense of shading/depth, you can render with a shaded brush to very loosely emulate lighting on spheres. This works because everything in your scene is a sphere and you're presumably not worried about things like shadows. All of this would be fairly easy to do in WPF (including the Shaded Brush), but be sure to use DrawingVisual classes and not framework elements. Also, you'll need to make sure you draw in the correct Z order, so it helps if you store the transformed points in a data structure that sorts as you add.
If you're still having performance problems, there are further optimizations you can pursue. For instance, if you know that only a subset of your points are moving, you can cache the transformed locations for the immobile points. It really depends on the nature of your data set and how it evolves.
Since your data set is so large, you might consider changing the way you visualize it. Instead of rendering 100K points, partition your working space into a volumetric grid and record the number (density) of points inside each grid cube. You can Project only the center of the grid and render it as a 'sphere' with some additional feedback (like color, opacity or brush texture) to indicate the point density. You can combine this technique with the traditional rendering approach, by rendering near points as 'spheres' and far points as 'cluster' objects with some brush patterning to match the density. One simple algorithm is to consider a bounding sphere around the camera; all points inside the sphere will be transformed normally; beyond the sphere, you will only render using the density grid.

Maybe the XNA Game studio is what you are looking for.
Also take a look at DirectX.

WPF perspective looked cool, but it seems targeted at static XAML defined scenes
Look again, WPF can be as dynamic as you will ever need.
You can write any WPF program, including 3D, totally without XAML.

Do you have to use C#/.Net or would MonoDevelop be good enough? I can recomend http://unity3d.com/ if you want a powerful 3D-engine.