Develop Reference

Pinch and rotate around a point using MRTK and Hololens 1

I'm trying to rotate a beam/cuboid around a pivot using MRTK, Unity, and the Hololens 1 when you're doing the pinch and hold gesture. The beam should remain in place once you've let go of the pinch.
My initial thoughts were to get the cartesian coordinates of the pinch and based on their position relative to the pivot, have the beam rotate by however many degrees needed. E.g. the hand position while pinching is (1,1,0), and the pivot position is (0,0,0). Thus, the beam should be rotated at 45 deg in the XY plane (we ignore the z components). I'm not sure how to go about doing this as the documentation seems to indicate that the only way to get the coordinates of the hand/pinch only works for the Hololens 2. (https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/Input/HandTracking.html#hand-tracking-events & https://microsoft.github.io/MixedRealityToolkit-Unity/api/Microsoft.MixedReality.Toolkit.Input.IMixedRealityHand.html#Microsoft_MixedReality_Toolkit_Input_IMixedRealityHand_TryGetJoint_).
Does anyone know how to go about doing this or at least point me in the right direction (tutorials/code/assets would be much appreciated!)
Thank you!

I'm not sure how to go about doing this as the documentation seems to indicate that the only way to get the coordinates of the hand/pinch only works for the Hololens 2
Yes, HoloLens1 does not support hand tracking, such as touching holograms directly with your hands or pointing and committing with hands. It is recommended that you try to use the interaction model Gaze and commit, so that you can easily get the position of GGVPointer.

Pinch to rotate interaction can be achieved by adding the ManipulationHandler component from MRTK to your cube. The component can be configured to allow two hand manipulation like this.
I'm not sure how to go about doing this as the documentation seems to indicate that the only way to get the coordinates of the hand/pinch only works for the Hololens 2.
There are a few ways to query pointer position. The code below should return the Right GGVPointer position for the hololens.
Vector3 pos;
GGVPointer pointer = PointerUtils.GetPointer<GGVPointer>(Handedness.Right);
if(pointer != null)
{
pos = pointer.Position;
}

In case you just want to rotate the Object around its center,you can use the Boundingbox Component. It creates handles that can be pinched and moved to rotate an object. you can disable the axis you don't want. It works even on the HoloLens 1.

How to convert the bounds of a box to screen coordinates

I have a 2D unity project.
I cannot depend on OnMouseExit because overlapping 2D box colliders cause the method to trigger even when the mouse is inside the bounds, since something else is in front (which is not my intention).
I was going to manually check for the mouse exiting on every frame by using:
if(!_collider.bounds.contains(Input.MousePosition))
But this does not work because `mouse position' is in terms of the number of pixels across the screen, and 'bounds' is in terms of "units" relative to the origin of the scene. The camera is Orthographic and slides around to look at the 2D plane that the world's sprites sit on. I have no idea how many "units" fit across the screen and suspect that it would change as soon as you change the aspect ratio or screen size.

You can use ScreenToWorldPoint(), to convert from screen point to 3D/2D point based on the camera's viewport, something like this:
if(!_collider.bounds.contains(Camera.main.ScreenToWorldPoint(Input.MousePosition)))

Collision depending of side of the rectangle?

Im not intrested in the code for this problem i just want to be pointed in the right direction.
Im using C# XNA if that helps.
Basicaly for my game i am adding collisions for example the player can't walk or fall through stones.. I have the rectangle and i know i use the .intersects comand but would i need to check collision depending on the side of the rectangle here?
I know if the player is falling i can make if playerRectangle intersects stoneRectangle playerY = stopFalling..
But if the rectangle is coliding on the side of the player.. That would be different wouldn't it ?

One simple way of dealing with this is to work out the main axis of penetration, i.e. is the player further into the rectangle horizontally, or vertically. Then, using this information, you'd move the player so they were just touching the rectangle.
i.e. Player bumps into a wall on his right.
Main axis is Horizontal, and he's to the left of the centre of the rectangle, so we know to move him left.
Searching on gamedev.stackexchange.com will give you lots of different collision detection/response options.

If your game is 2D I would recommend using something like FarSeer Physics rather than implementing yourself.
If you are interested in the algorithms it is open source and you can poke around the code to see how they implemented collision detection.

Rotating part of an image in 3D space

Here's the setup: This is for an ecommerce art site where some paintings are canvas transfers. The painting wraps around the sides and top and bottom of the canvas. We have high-res images of the entire painting, but what we want to display is a quasi-3D representation of the image in which you can see how the sides of the painting wrap around the canvas. Here's a rough sketch of what I'm talking about:
My question is, how can I rotate an image in 3D space? The approach I think I'd like to take, is to cut off a portion of the top and side of the image, and rotate then in 3D and then stich it back on to the top and side to give it the 3D look. How do I go about about doing that? It can be done using any .Net technology (GDI+, WPF etc.).

In WPF using the ViewPort3D class you can create a cuboid which is 8x5x1 units. Create the image as a texture and then apply the texture to the front face (8x5) and the side faces (5x1) and the top and bottom faces (8x1) using texture coordinates. The front face coordinates should be: (1/9, 1/6), (8/9, 1/6), (1/9, 5/6) and (8/9, 5/6) for the front face, and from the nearest edge to those coordinates for the sides, e.g. for the left side: (0, 1/6), (1/9, 1/6), (0, 5/6) and (1/9, 5/6) for the left side.
Edit:
If you then want to be able to perform rotations on the 3D canvas model you can follow the advice here:
How can I do 3D transformation in WPF?

It looks like you're not needing to do real 3D, but only needing to fake it.
Chop off four strips along the top, bottom, left and right of the image. Toss the bottom and right (going by your sketch in the question). Scale and shear the strips (I'm not expert enough at .net/wpf to know how, but it can do it). The top would be scaled vertically by a factor of 0.5 (a guess - choose to fit the desired final 3D-looking image) and sheared horizontally. The result is composited onto the output image as the top side of the canvas. The left strip would be scaled horizontally and sheared vertically.
If the end user is to view the 3D canvas from different angles interactively, this method is probably faster than rendering an honest 3D model, which would have to do texture mapping and rasterizing the model into a final image, which amounts to doing the same math. The fun part is figuring out how to adjust the scaling and shearing parameters.
This page might be educational: http://www.idomaths.com/linear_transformation.php
and this could be useful reference http://en.csharp-online.net/GDIplus_Graphics_Transformation%E2%80%94Image_Transformation

I dont have any experience in this kind of stuff. But when i saw this question, the first thing comes to my mind is the funny Unicornify for SO.
In this making of article by balpha, he explained how the 2d unicorn sphere is rotated in 3d space.
But the code is written in python. If you are interested, you can take a look into that. But am not exactly sure this would help you.

The brute force approach (which might be the easiest approach), is to map the u,v texture coordinates for each of the three faces, onto three billboards representing three sides of the canvas (a billboard is just two triangles that make a rectangle). Then, rotate the whole canvas (all three billboards) using matrix transforms. Tada!
Alternately, you can move the 3-space camera position with a transform, rather than the canvas. Six of one, half the other - as they say.

Hiding objects that obscure the player in a 3D scene

I'm designing a 3D game with a camera not entirely unlike that in The Sims and I want to prevent the player character from being hidden behind objects, including walls, pillars and other objects.
One easy way to handle the walls case is to have them face inwards and not have an other side, but that won't cover the other cases at all.
What I had planned is to somehow check for objects that are "in front" of the player, relative to the camera, and hide them - be it by alpha blending or not rendering at all.
One probably not so good idea I had in mind is to scan from the camera to the player in a straight line and see if you hit a non-hidden object, continuing until you reach the player. Unfortunately, I am an almost complete newbie on 3D programming.
Demonstration SVG illustration < that wall panel obscures the player, so it must be hidden. Another unrelated and pretty much already solved problem is removing all three wall panels on that side, which is irrelevant to this question and only caused by the mapping system I came up with.

What I had planned is to somehow check for objects that are "in front" of the player, relative to the camera, and hide them - be it by alpha blending or not rendering at all.
This is a good plan. You'll want to incorporate some kind of bounding volume onto the player, so the entire player (plus a little extra) is visible at all times. Then, simply run the intersection algorithm for each corner of the bounding volume.

Finding which object is at a given point on screen is called picking. Here's an XNA link for you which should direct you to an example. The idea is that you retrieve the 3D point in the game from the 2D point, and then can use standard collision detection methods to work out which object is occupying that space. Then you can elect to render that object differently.
One hack which might suffice if you have trouble with the picking approach is to render the character once as part of the scene, and then render it again at the end at half-alpha on top of everything. That way you can see the whole character and the wall, though you won't see through the wall as such.

One easy way, at least for prototyping, would be to always draw the player after you draw the rest of the scene. This would ensure that the player is rendered on top of anything else in the scene. Crude but effective.

Create a bounding volume from the camera to the extents of the player, determine what objects intersect that volume, and then render them in whatever alternate style you want?
There might be some ultra-clever way to do this, but this seems like the pretty straightforward version, and shouldn't be too much of a perf hit (you're probably doing collision every frame anyway....)

The simplest thing I can think of that should work relatively well is to model all obstacles by a plane perpendicular to your ground (assuming you have a ground.) Roughly assuming everything that is an obstacle is a wall with some height.
Model your player as a point somewhere, and model your camera as another point. The line in 3d that connects these two points lies in a plane that is particularly interesting to you, because if this plane intersects an "obstacle plane" below the height of the obstacle, that means that that obstacle is blocking your view of the player point.
I hope thats somewhat clear. To make this into an algorithm you'd have to implement a general method for determining where two planes intersect (to determine if the obstacle is tall enough to block view.)