I have a 360 spherical video. I use this video as a texture on a sphere in Unity. Inside the sphere is a camera and this functions as the setup for my Virtual Reality experience. Pretty basic.
I am trying to write a bit of code on the web where people can upload 360 images and videos, place a marker/hotspot on the 360 spherical image/video, and then apply the image/video-texture on the sphere in Unity3D. If I overlay a simple x/y coordinate grid on the 360 video/image-texture, put in some x/y-coordinates to place the marker/hotspot, and put the texture back on the sphere, Unity will not interpret this correctly since we are now in 3D space and we are looking at the texture from within the sphere mapped onto the plane with all the distortion happening.
My question is, how do I convert these x and y coordinates on the 2D plane of the 360 video texture to coordinates that can be understood in 3D within Unity3D?
My first thought was to use 2-dimensional cartesian coordinates and convert these into spherical coordinates, but I seem to be missing a z-axis in the cartesian coordinates to make this work.
Is the z-axis simply 0 or is it the radius from center of the sphere to the x/y-coordinate? What does the z-axis represent? Is there maybe two coordinate systems. One that is coordinates on a plane and one that is from the centre of the sphere?
This is the conversion code that I have so far:
public static void CartesianToSpherical(Vector3 cartCoords, out float outRadius, out float outPolar, out float outElevation){
if (cartCoords.x == 0)
cartCoords.x = Mathf.Epsilon;
outRadius = Mathf.Sqrt((cartCoords.x * cartCoords.x)
+ (cartCoords.y * cartCoords.y)
+ (cartCoords.z * cartCoords.z));
outPolar = Mathf.Atan(cartCoords.z / cartCoords.x);
if (cartCoords.x < 0)
outPolar += Mathf.PI;
outElevation = Mathf.Asin(cartCoords.y / outRadius);
}
This is my very first post so please excuse me if I am doing anything wrong and let me know how to improve.
Spherical co-ordinates are different from 2d or 3d co-ordinates as you need to measure them in radians. It is usually measured from the center of the sphere and XY axis. It marks a rectangular area on the surface of the sphere. Please refer to this link for spherical co-ordinates in unity - https://blog.nobel-joergensen.com/2010/10/22/spherical-coordinates-in-unity/
Related
Although my current project is XNA, this question is about the basic mathematics of 3D to 2D mapping. In fact, and for the same reason, let's assume a WinForms graphics surface to draw on.
I have the following configuration:
Camera position of (x=0, y=0, z=0) and direction vector of (x=0, y=0, z=0).
A line segment in 3D with the following points: (10, 10, 10), (100, 100, 100).
I want to transform these coordinates and draw them on a 2D surface. So depending on the camera, the line segment should transform from (x1, y1, z1),(x2, y2, z2) to (x1, y1),(x2, y2).
I think you are looking for an orthogonal or perspective projection. There is a lot of information online if you search for it but here is the gist.
A camera looking at the origin, located a distance d along the z-axis will project a point at (x,y,z) onto a plane as:
// Orthogonal
planar_x = x
planar_y = y
// Perspective
planar_x = x*d/(d-z)
planar_y = y*d/(d-z)
Example
A point at (10,10,10) with the camera located a distance of 500 along the z axis will have planar coordinates (10*500/(500-10), 10*500/(500-10)) = (10.204, 10.204)
A point at (10,10,100) with the camera located a distance of 500 along the z axis will have planar coordinates (10*500/(500-100), 10*500/(500-100)) = (12.5, 12.5)
So the closer a shape is to the camera the larger it appears.
To transform the planar model coordinates to pixel coordinates I use the following scaling
scale = max_model_size/Math.Min(Height,Width);
pixel_x = Width/2 + x/scale;
pixel_y = Height/2- y/scale;
This is how I can use GDI to draw 3D shapes on a windows form.
Of course if you want to use OpenGL then look here for a similar question.
My main plane is Rectangle(0,0,10000,10000) for example.
My screen plane (ie virtual position) is Rectangle(1000,1000,1920,1080).
My Texture2D is Rectangle(1500,1200,200,100) in main plane.
I need to translate my Texture2D coordinates to my screen plane. I tried with Matrix.Translate without success.
I must get Texture2D = Rectangle(500,200,200,100) in screen plane.
In order to get the Texture2D from (1500, 1200) to (500, 200) you have to use a translation of (-1000, -1000) which are the inverse numbers from your screen plane's coordinates. In code your translation would be something like this
Matrix transform = Matrix.CreateTranslation(-screenPlane.x, -screenPlane.y, 0);
The theory is that you want to move the texture like if your camera was on (0, 0) instead of (1000, 1000). You have to move the texture by (-1000, -1000) in order to do so.
Check the web for 2D camera classes, always usefull to know how cameras work :)
This one for example: http://www.david-amador.com/2009/10/xna-camera-2d-with-zoom-and-rotation/
I'm trying to rotate a bounding box in xna, this is how I usually do it:
new Rectangle((int)position.X, (int)position.Y, (int)texture.Width, (int)texture.Height);
but It currently does not return the rotation that is in my draw code:
spriteBatch.Draw(texture, position, null, Color.White, rotation, origin, scale, SpriteEffects.None, 1f);
how do I rotate it with the player? I was trying to figure this out and stumbled across this:
2D BoundingRectangle rotation in XNA 4.0
but I'm not quite sure how to implement the Matrices he's talking about,and I'm making a 2d game as opposed to the 3d one here.
The Rectangle structure is an axis aligned bounding rectangle. The sides will always be parallel to the X and Y axes. If you want a rotated bounding rectangle you have to implement it yourself.
One algorithm for calculating collision of rotated rectangles is the Separating Axis Theorem.
Matrices are, as stated in the linked answer, essential in computer graphics. You can rotate a shape by rotating each of its vertices with the same rotation matrix. This is the general rotation matrix for two dimensions:
Matrices are then "applied" to vectors with matrix-vector multiplication:
which equals
I struggled with this problem for two days before finding the solution
Rotating bounding box
Hope this will help !
I have three Vector3 points in 3D space. I need to copy the rotation (the tangent?) of this triangle to the orientation of a 3D model. How can I calculate the triangles Vector3 tangent or create a rotation matrix out of those points?
Finding the angle of a triangle is described here: Find the normal angle of the face of a triangle in 3D, given the co-ordinates of its vertices
Suppose you find the normal and call it N'. It should be trivial for you to write the normal of the "unrotated" triangle, N, eg <1, 0, 0>. It should also be trivial to figure out how to rotate from N to N' and you can create a rotation matrix for it with Matrix.CreateFromAxisAngle in XNA. This matrix should rotate everything like you want.
So far my process is as follows:
Markers are square markers of a known size, an example is below:
Detect markers
Get translation + rotation of markers relative to camera using POSIT - result is a transformation matrix of the matrix in camera coordinate space.
Now I am trying to find the distance in mm between the related corners of two markers. An example is below:
By distance I mean the euclidean distance (sqrt(x^2 + y^2 + z^2)).
I am thinking it should be possible as I have a reference for mm distance (the edge of any of the markers is a known mm distance), but I am unsure about how I would get to this from just the two transformation matrices I have.
EDIT:
I have managed so far to get the relative distance between the marker centers by multiplying the 2nd transform matrix by the inverse of the first and looking at the translation of the result.
I figured that I can get the plane of each marker by getting the normal and point. From there I should be able to get the 3d coord of a corner by looking at how the 2D point intersects that plane and from there the rest should be easy. I am not entirely sure how to get the normal of the marker and how to work out the point where the 2D coord representing the corner intersects the plane.
Extra info:
Using XNA and C#
Using AForge.net for video input and marker detection.
For calculation simplicity, I will suppose that the markers are on the XY plane. Otherwise, you would have to do some projections to get the camera position into the marker's plane.
Given the translation matrix, you should be able to define the camera position. We will try to find the absolute difference between those point's coordinate
Let's assume pC(xc,yc) the camera position, pA(xa,ya) the first marker position, pB(xb,yb) the second marker position.
lAC the absolute distance between A and the camera, which is known
lBC the absolute distance between B and the camera, which is known
lAB the absolute distance between B and A
dAC(dcax,dcay) the manhattan distance between the camera and A,
dBC(dcbx,dcby) and dAB(dabx,daby) likewise.
dcax = sqrt( lAC^2 - (xc-xa)^2)
dcay = sqrt( lAC^2 - (yc-ya)^2)
dcbx = sqrt( lBC^2 - (xc-xb)^2)
dcby = sqrt( lBC^2 - (yc-yb)^2)
dabx = abs(dcbx-dcax)
daby = abs(dcby-dcay)
lAB = sqrt(dabx^2 + daby^2)
Ps: It's been a long time since I've done this, Would appreciate a double check :)