Kinect Cursor Control with virtual XNA-Rectangle-'Touchpad' - Y-Axis inverted - c#

I'm trying to implement a Real-Time Strategy control scheme for the MS Kinect.
So far, I've got a cursor, which can be moved by moving your left Hand (or right, dependant on your handedness). I've got an Open-NI-based Kinect controller which sets up a skeleton for player-movements and delivers the wrist-, elbow-, shoulder- and body-center-coordinates to my application.
To project these wrist-coordinates to the screen, I've set up a Rectangle, which is situated slightly left/right from the player's center and as long as the wrist moves inside the rectangle, the cursor moves on screen.
My problem is, that the XNA-Rectangle has the upper left corner as point of origin, i.e. the X-axis points right, as it "should", but the Y-axis points down, while the Y-axis of the Kinect - coordinate system points up. This results in the cursor moving upwards on screen, when I move my hand down and vice versa. There's no way for me to change anything with the Kinect-coordinate system, so is it possible to 'flip' the 'coordinate system' of the rectangle, so that it's Y-axis points up,too?
Here's the relevant code:
(from Calibrate()-Method:)
List<Vector3> joints = UDPlistener.getInstance().ParseCalibCoordinates(data);
//0 = Right Wrist 1 = Right Elbow 2 = Right Shoulder
//3 = Left Wrist 4 = Left Elbow 5 = Left Shoulder
//6 = Center
height = 762;
width = 1024;
switch (hand)
{
case 0:
cursorSpace = new Rectangle((int)(joints[6].X * 2) - 200, (int)(joints[6].Y * 2) + height, width, height);
break;
case 3:
cursorSpace = new Rectangle((int)(joints[6].X * 2) - 1200, (int)(joints[6].Y * 2) + height, width, height);
break;
}
public Point Cursor(String data)
{
List<Vector3> joints = UDPlistener.getInstance().ParsePlayCoordinates(data);
//0 = Right Wrist 1 = Left Wrist 2 = Center
double mhx = 0; //main hand x-coordinate
double mhy = 0; // main hand y-coordinate
switch (hand)
{
case 0:
mhx = joints[hand].X;
mhy = joints[hand].Y;
break;
case 3:
mhx = joints[hand-2].X;
mhy = joints[hand-2].Y;
break;
}
int x;
int y;
if (Math.Abs(mhx - mhxOld) < 1.0 || Math.Abs(mhy - mhyOld) < 1.0)
//To remove jittering of the cursor
{
x = (int) mhxOld * 2;
y = (int) mhyOld * 2;
}
else
{
x = (int) mhx * 2;
mhxOld = mhx;
y = (int) mhy * 2;
mhyOld = mhy;
}
Point cursor = new Point(0,0);
if (cursorSpace.Contains(x,y))
{
cursor = new Point(x - cursorSpace.X, y - CursorSpace.Y);
lastCursorPos = cursor;
return cursor;
}
Sorry for the wall of text, I hope, I could make myself clear.
Thanks in advance,
KK

I use an extension method for converting OpenNI coordinates. The following example maps the OpenNI coordinates to XNA coordinates in a 640x480 rectangle in the top left corner, represented as a Vector2 object.
public static Vector2 ToXnaCoordinates(this Point3D point)
{
return new Vector2(
point.X + 320,
(point.Y - 240) * -1);
}
The magic that flips the y coordinate is the * -1 part.
If you want to reach a rectangle of different size than 640x480, you need to scale the coordinates accordingly after conversion. Example:
public static Vector2 ToScaledXnaCoordinates(this Point3D point, int rectSizeX, int rectSizeY)
{
return new Vector2(
(point.X + 320) * rectSizeX / 640,
(point.Y - 240) * -rectSizeY / 480);
}

I know this isn't XNA, but I wanted to put this out there for those wpf users:) If you are using something like Channel 9's approach, just have a bool to determine if inverted or not. Example:
private void ScalePosition(FrameworkElement element, Joint joint, bool inverted)
{
//convert the value to X/Y
Joint scaledJoint = joint.ScaleTo(967, 611);
//convert & scale (.3 = means 1/3 of joint distance)
//Joint scaledJoint = joint.ScaleTo(1280, 720, 1f, 1f);
if (!inverted)
{
Canvas.SetLeft(element, scaledJoint.Position.X);
Canvas.SetTop(element, scaledJoint.Position.Y);
}
if (inverted)
{
Canvas.SetLeft(element, scaledJoint.Position.X);
Canvas.SetBottom(element, scaledJoint.Position.Y);
}
}
Hope this helps WPF users!

Related

How to anchor unity game object to the corner of a cameras view

I need to anchor a game object to the corner of a cameras view, regardless of that objects size. I need to do this because I am making UI without unity's built in system (the built in one messes with my pixel art).
Here's the code I've come up with thus far:
`
public class BuildingMenuScalePos : MonoBehaviour
{
private Transform myTransform;
int screenX;
int screenY;
int offSet;
float prevCamScale;
float camScale;
Vector2 screenSize;
Vector2 prevScreenSize;
// Start is called before the first frame update
void Start()
{
prevCamScale = Camera.main.orthographicSize;
myTransform = GetComponent<Transform>();
}
// Update is called once per frame
void Update()
{
// Updates anchor posisiton when camera scale or screen size changess
// Important because pixel perfect camera changes orthographic size at the start
camScale = Camera.main.orthographicSize;
screenSize = new Vector2(Screen.width, Screen.height);
offSet = ((Screen.height / 180) / 2) * 8;
if (camScale != prevCamScale || screenSize != prevScreenSize)
{
myTransform.localScale = new Vector3(8.5f, (float)Screen.height / (offSet * 2), 1);
setUIToAnchor(2, -offSet * (float)myTransform.lossyScale.x, -offSet * (float)myTransform.lossyScale.y);
prevCamScale = camScale;
prevScreenSize = screenSize;
}
}
// Moves object to anchor point
void setUIToAnchor(int anchorPos, float offSetX, float offSetY)
{
Vector3 UIWorldPos = Camera.main.ScreenToWorldPoint(setAnchor(anchorPos, offSetX, offSetY)); // Converts from pixels to world space
myTransform.position = UIWorldPos;
}
// Sets object anchor position on screen (in pixels)
Vector3 setAnchor(int anchorPos, float offSetX, float offSetY)
{
switch (anchorPos)
{
case 0: // Bottom Left
screenX = 0;
screenY = 0;
break;
case 1: // Top Left
screenX = 0;
screenY = Screen.height;
break;
case 2: // Top Right
screenX = Screen.width;
screenY = Screen.height;
break;
case 3: // Bottom Right
screenX = Screen.width;
screenY = 0;
break;
case 4: // Left side middle
screenX = 0;
screenY = Screen.height / 2;
break;
case 5: // Up side middle
screenX = Screen.width / 2;
screenY = Screen.height;
break;
case 6: // Right side middle
screenX = Screen.width;
screenY = Screen.height / 2;
break;
case 7: // Down side middle
screenX = Screen.width / 2;
screenY = 0;
break;
case 8: // Center
screenX = Screen.width / 2;
screenY = Screen.height / 2;
break;
default:
break;
}
return new Vector3(screenX + offSetX, screenY + offSetY, 1);
}
}
`
This code gets anchor position in pixels, then converts that number into world space units. Afterwards it just places my UI game object at those world space coordinates.
The code works, but only using Free aspect, 4:3, 16:9, etc. If I use a set resolution like 1920x1080 the box is no longer placed in the corner of the screen correctly. The offset seems to cause this behavior, because if I set it to 0 the middle of the UI sprite is placed perfectly in the corner regardless of res or aspect ratio. Set resolution also seem to break the pixel perfect camera, because my sprites become blurry.
Screenshot examples
I just want to know why free aspect ratios and fixed res are different. And I'd appreciate if someone knows how to make pixel perfect UI, that's maybe a more elegant solution than what I'm doing.

How do you translate chart coordinates to device pixels in SkiaSharp?

I'm attempting to write a matrix transform to convert chart points to device pixels in SkiaSharp. I have it functional as long as I use 0,0 as my minimum chart coordinates but if I need to to step up from a negative number, it causes the drawing to shift left and down. That is to say that the X Axis is shifted to the left off the window and the Y Axis is shift down off the window.
This is intended to be a typical line chart (minimum chart point at the lower left while minimum device point at the upper left). I have accounted for that already in the transform.
While stepping through code I can see that the coordinates returned from the Matrix are not what I expect them to be, so I believe the issue to be with my transform but I haven't been able to pinpoint it.
UPDATE: After further examination, I believe I was mistaken, it is not shifted, it's just not scaling properly to the max end of the screen. There is a bigger margin at the top and right side of the chart than there should be, but the bottom and left side are fine. I've been undable to determine why the scaling doesn't fill the canvas.
Below are my matrix methods:
private SKMatrix ChartToDeviceMatrix, DeviceToChartMatrix;
private void ConfigureTransforms(SKPoint ChartMin,
SKPoint ChartMax, SKPoint DeviceMin, SKPoint DeviceMax)
{
this.ChartToDeviceMatrix = SKMatrix.MakeIdentity();
float xScale = (DeviceMax.X - DeviceMin.X) / (ChartMax.X - ChartMin.X);
float yScale = (DeviceMin.Y - DeviceMax.Y) / (ChartMax.Y - ChartMin.Y);
this.ChartToDeviceMatrix.SetScaleTranslate(xScale, yScale, DeviceMin.X, DeviceMax.Y);
this.ChartToDeviceMatrix.TryInvert(out this.DeviceToChartMatrix);
}
// Transform a point from chart to device coordinates.
private SKPoint ChartToDevice(SKPoint point)
{
return this.ChartToDeviceMatrix.MapPoint(point);
}
The code invoking this is:
void OnCanvasViewPaintSurface(object sender, SKPaintSurfaceEventArgs args)
{
SKImageInfo info = args.Info;
SKSurface surface = args.Surface;
SKCanvas canvas = surface.Canvas;
float strokeWidth = 1;
float margin = 10;
// SKPaint definitions omitted for brevity.
var ChartMin = new SKPoint(-10, -1); // Works fine if I change this to 0,0
var ChartMax = new SKPoint(110, 11);
var DeviceMin = new SKPoint(margin, margin);
var DeviceMax = new SKPoint(info.Width - margin, info.Height - margin);
const float stepX = 10;
const float stepY = 1;
const float tickX = 0.5;
const float tickY = 0.075F;
// Prepare the transformation matrices.
this.ConfigureTransforms(ChartMin, ChartMax, DeviceMin, DeviceMax);
// Draw the X axis.
var lineStart = new SKPoint(ChartMin.X, 0);
var lineEnd = new SKPoint(ChartMax.X, 0);
canvas.DrawLine(this.ChartToDevice(lineStart), this.ChartToDevice(lineEnd), axisPaint);
// X Axis Tick Marks
for (float x = stepX; x <= ChartMax.X - stepX; x += stepX)
{
var tickMin = new SKPoint(x, -tickY);
var tickMax = new SKPoint(x, tickY);
canvas.DrawLine(this.ChartToDevice(tickMin), this.ChartToDevice(tickMax), axisPaint);
}
// Draw the Y axis.
// The inversion of above, basically the same.
I was able to discover my own problem with enough time. I wasn't calculating the offset correct.
this.ChartToDeviceMatrix.SetScaleTranslate(xScale, yScale, DeviceMin.X, DeviceMax.X);
Should have been:
this.ChartToDeviceMatrix.SetScaleTranslate(xScale, yScale, -ChartMin.X * xScale + DeviceMin.Y, -ChartMin.Y * yScale + DeviceMax.Y);
Final Matrix method was:
private SKMatrix ChartToDeviceMatrix, DeviceToChartMatrix;
private void ConfigureTransforms(SKPoint ChartMin, SKPoint ChartMax, SKPoint DeviceMin, SKPoint DeviceMax)
{
this.ChartToDeviceMatrix = SKMatrix.MakeIdentity();
float xScale = (DeviceMax.X - DeviceMin.X) / (ChartMax.X - ChartMin.X);
float yScale = (DeviceMin.Y - DeviceMax.Y) / (ChartMax.Y - ChartMin.Y);
float xOffset = -ChartMin.X * xScale + DeviceMin.X;
float yOffset = -ChartMin.Y * yScale + DeviceMax.Y;
this.ChartToDeviceMatrix.SetScaleTranslate(xScale, yScale, xOffset, yOffset);
this.ChartToDeviceMatrix.TryInvert(out this.DeviceToChartMatrix);
}

Lines and shapes drawn at incorrect angles

I have a c# program where I need to draw some simple 2D objects on the canvas.
One of these involves drawing a rectangle and lines where I know the start point, the length and I have to calculate the end position. So I have the following code;
private void CalculateEndPoint()
{
double angle = Helper.deg2rad((double)this.StartAngle);
int x = this.StartPoint.X + (int)(Math.Cos(angle) * this.Length * -1);
int y = this.StartPoint.Y + (int)(Math.Sin(angle) * this.Length);
this.EndPoint = new Point(x, y);
}
Now this seems to work OK to calculate the end points. The issue I have is with the angle (this.StartAngle), the value I specify seems not to be how it is drawn and I seem to have the following;
Where as I'm expecting 0 at the top, 90 on the right, 180 at the bottom etc.
So to get a shape to draw straight down the canvas I have to specify 90 degrees, where as I would expect to specify 180.
Have I done something wrong? Or is it just a lack of understanding?
You should change your CalculateEndPoint function to have that:
private static void CalculateEndPoint(double dec)
{
double angle = (Math.PI / 180) * (this.StartAngle + 90); // add PI / 2
int x = StartPoint.X + (int)(Math.Cos(angle) * Length * -1);
double angle2 = (Math.PI / 180) * (this.StartAngle - 90); // minus PI / 2
int y = StartPoint.Y + (int)(Math.Sin(angle2) * Length);
EndPoint = new Point(x, y);
}
Actually, 0 should be on the right. You are multiplying the x-coordinate by -1, so you're moving it to the left.
Just remember these 2 rules:
- The cosine of the angle is the x-coordinate of the unit circle.
- The sine of the angle is the y-coordinate of the unit circle.
Since cos(0) = 1 and sin(0) = 0, the coordinate corresponding to angle 0 is (1, 0).
Whether 90 is on top or on the bottom depends on the canvas.
Some applications/frameworks consider y-coordinate 0 to be at the top of the canvas. That means you go clockwise around the circle and 90 will be at the bottom.
If y-coordinate 0 is at the bottom of the canvas, you go counter-clockwise and 90 will be at the top.

Drawing triangles given user input

So I'm trying to make a Program that will draw a triangle given some user input. The variables that the user provides are angleA, angleB, andleC, and the corresponding sides. The code I have set up to find the three points of the angle is as follows.
double angle_A = double.Parse(angleA.Text);
double angle_B = double.Parse(angleB.Text);
double angle_C = double.Parse(angleC.Text);
double side_A = double.Parse(sideA.Text);
double side_B = double.Parse(sideB.Text);
double side_C = double.Parse(sideC.Text);
double triangleHeight = Area * 2 / (double.Parse(sideB.Text));
double height = canvas.Height;
double width = canvas.Width;
int aX, aY, bX, bY, cX, cY;
aY = Convert.ToInt32(canvas.Height - triangleHeight / 2);
if (angle_A <= 90 && angle_C <= 90)
{
aX = Convert.ToInt32((width - side_B) / 2);
}
else if (angle_A > 90)
{
double extraLength = Math.Sqrt(Math.Pow(side_C, 2) - Math.Pow(triangleHeight, 2));
aX = Convert.ToInt32(width - ((width - (side_B + extraLength)) / 2) + side_B);
}
else if (angle_C > 90)
{
double extraLength = Math.Sqrt(Math.Pow(side_A, 2) - Math.Pow(triangleHeight, 2));
aX = Convert.ToInt32((width - side_B + extraLength) / 2);
}
else
{
aX = 0;
MessageBox.Show("ERROR: No such triangle exists", "ERROR:");
}
cX = aX + Convert.ToInt32(side_B);
cY = aY;
bX = Convert.ToInt32(side_A * Math.Cos(Math.PI * angle_C / 180) + cX);
bY = Convert.ToInt32(side_A * Math.Sin(Math.PI * angle_C / 180) - cY);
Point pointA = new Point(aX, aY);
Point pointB = new Point(bX, bY);
Point pointC = new Point(cX, cY);
Point[] points = new Point[3] { pointA, pointB, pointC };
return points;
This returns the three points that the paint method should use to draw the triangle. However, when I insert the values, the triangle it draws looks nothing like the triangle I have described with the user input. Any thoughts on why this is? Thanks in advance.
P.S. The error is not in my code, as it gives me no errors and does not crash. It is strictly a math error that I have not been able to locate.
I imagine the triangle ABC with corners A and C along the base line with A to the left and C to the right, and B somewhere above them. Side A is the side opposite corner A, and so on.
As Damien_the_Unbeliever says, you should only allow input of, say, side B, side C and angle of corner A. Validate that A is not over 180 degrees. Start off with A at the origin, so we know straight away that xA = 0, yA = 0, xC = length of side B, yC=0, xB = side C * cos A, and yB = side C * sin A. I believe this works even if A is over 90 degrees, you do get a negative value for xB but don't worry, continue anyway!
Now all you have to do is centre the triangle on the canvas. I don't understand where you are getting Area from. It makes no sense to calculate the triangle's height from its area. The triangle height is yB, you can calculate the offset you need to centre it vertically as you know, so long as yB <= height. Add the same y offset to all the points.
The horizontal offset is a bit more complicated! If xB is negative, I would add an offset to all the x values to bring xB to 0, this positions your triangle at the left side of the canvas, and its width is given by the new xC. If xB is non-negative, the width is the maximum of xC or xB. Then you can calculate the x offset from the width as you know.
I have had time to do some of the code, for example values; this will draw a triangle but not yet centre it:
int sideB = 100;
int sideC = 143;
int angleA = 28;
double angleARadians = Math.PI * angleA / 180.0;
int[] xs = new int[3];
int[] ys = new int[3];
//1st corner is at the origin
xs[0] = 0; ys[0] = 0;
//Then the third corner is along the x axis from there to the length of side B
xs[2] = sideB; ys[2] = 0;
// The second corner is up a bit above the x axis. x could be negative.
// Note, when you draw it, the y axis extends downwards, so a positive y value will be drawn below the x axis.
xs[1] = (int)Math.Round(sideC * Math.Cos(angleARadians));
ys[1] = (int)Math.Round(sideC * Math.Sin(angleARadians));
//If Bx is negative, move all the points over until it's 0
if (xs[1] < 0)
{
int zeroX = xs[1] * -1;
for (int i = 0; i < 3; i++)
{
xs[i] += zeroX;
}
}
// Now centre the triangle on the canvas.
// Firstly find the width of the triangle. Point B could be to the left of A, or between A and C, or to the right of C.
// So the left most point of the triangle is the minimum of A or B, and the right most point is the maximum of B or C.
int minX = Math.Min(xs[0],xs[1]);
int maxX = Math.Max(xs[2], xs[1]);
//The height of the triangle is yB.
int offsetX = (panCanvas.Width - (maxX - minX)) / 2;
int offsetY = (panCanvas.Height - ys[1]) / 2;
//offset all the points by the same amount, to centre the triangle.
for (int i = 0; i < 3; i++)
{
xs[i] += offsetX;
ys[i] += offsetY;
}
Given the three sides of a triangle a, b, and c the coordinates of the vertices are
P=[0,0]
Q=[a,0]
R=[(a^2+c^2-b^2)/(2*a), sqrt(c^2*(2*(a^2+b^2)-c^2)-(a+b)^2*(a-b)^2)/(4*a^2))]
Example, a=6, b=4 and c=8
P=[0,0]
Q=[6,0]
R=[7,√15]

Problems with centering scene

I have to display stl models with openGL. (SharpGL.) I'd like to set the initial view, so that the model is at the center of the screen and approximately fills it. I've calculated the bounding cube of the models and set the view like this: (sceneBox is a Rect3D - it stores the location of the left-back-bottom corner and the sizes)
// Calculate viewport properties
double left = sceneBox.X;
double right = sceneBox.X + sceneBox.SizeX;
double bottom = sceneBox.Y;
double top = sceneBox.Y + sceneBox.SizeY;
double zNear = 1.0;
double zFar = zNear + 3 * sceneBox.SizeZ;
double aspect = (double)this.ViewportSize.Width / (double)this.ViewportSize.Height;
if ( aspect < 1.0 ) {
bottom /= aspect;
top /= aspect;
} else {
left *= aspect;
right *= aspect;
}
// Create a perspective transformation.
gl.Frustum(
left / ZoomFactor,
right / ZoomFactor,
bottom / ZoomFactor,
top / ZoomFactor,
zNear,
zFar);
// Use the 'look at' helper function to position and aim the camera.
gl.LookAt(
0, 0, 2 * sceneBox.SizeZ,
sceneBox.X + 0.5 * sceneBox.SizeX, sceneBox.Y + 0.5 * sceneBox.SizeY, sceneBox.Z - 0.5 * sceneBox.SizeZ,
0, 1, 0);
This works nice with my small, hand-made test model: (it has a box size of 2*2*2 units)
This is exactly what I want. (The yellow lines show the bounding box)
But, when I load an stl model, which is about 60*60*60 units big, I get this:
It's very small and too far up.
What should I change to make it work?
Here's the full thing: https://dl.dropbox.com/u/17798054/program.zip
You can find this model in the zip as well. The quoted code is in KRGRAAT.SZE.Control.Engine.GLEngine.UpdateView()
Apparently the problem are the arguments you are using in lookAt function. If you have calculated bounding cube all you need to do is to place it in the distance (eyeZ) from the camera of
sizeX/tan(angleOfPerspective)
where sizeX is width of Quad of which cube is built, angleOfPerspective is first parameter of GlPerspective of course having centerX == posX == centreX of the front quad and centerY == posY == centreY of the front quad and frustum is not necessary
lookAt reference http://www.opengl.org/sdk/docs/man2/xhtml/gluLookAt.xml
So, to clarify Arek's answer, this is how I fixed it:
// Calculate viewport properties
double zNear = 1.0;
double zFar = zNear + 10 * sceneBox.SizeZ; // had to increase zFar
double aspect = (double)this.ViewportSize.Width / (double)this.ViewportSize.Height;
double angleOfPerspective = 60.0;
double centerX = sceneBox.X + 0.5 * sceneBox.SizeX;
double centerY = sceneBox.Y + 0.5 * sceneBox.SizeY;
double centerZ = sceneBox.Z + 0.5 * sceneBox.SizeZ;
// Create a perspective transformation.
gl.Perspective( // swapped frustum for perspective
angleOfPerspective / ZoomFactor, // moved zooming here
aspect,
zNear,
zFar);
// Use the 'look at' helper function to position and aim the camera.
gl.LookAt(
centerX, centerY, sceneBox.SizeX / Math.Tan(angleOfPerspective), // changed eye position
centerX, centerY, -centerZ,
0, 1, 0);

Categories

Resources