Unity3D & Realsense 2.0 D415 - Depth to Terrain - c#

I am trying to generate a runtime terrain from the depth values of the RealSense camera D415.
I got the camera to try and do something for the summer. I never used more than webcams in Unity before so I'm very lost even after going through all the sample scenes.
My terrain is 500 x 500 x 40
using RealSense SDK 2.0
the terrain generator code is below which works with Perlin noise
using UnityEngine;
using Intel.RealSense;
/// <summary>
/// this script is made to test the new terrain in unity with intel realsense depth
/// </summary>
public class TerrainGenerator : MonoBehaviour {
public int depth = 40;
public int width = 500;
public int height = 500;
public float scale = 20f;
public float xOffset = 10;
public float yOffset = 10;
private DepthFrame depthFrame;
public bool scroll;
public float scrollSpeed;
private Pipeline pipe;
private void Start()
{
///
pipe = new Pipeline();
pipe.Start();
///
xOffset = Random.Range(0f, 9999f);
yOffset = Random.Range(0f, 9999f);
scroll = false;
scrollSpeed = 0;
}
void Update()
{
Terrain terrain = GetComponent<Terrain>();
terrain.terrainData = GenerateTerrain(terrain.terrainData);
if(scroll)
xOffset += Time.deltaTime * scrollSpeed;
}
TerrainData GenerateTerrain(TerrainData tData)
{
tData.heightmapResolution = width + 1;
tData.size = new Vector3(width, depth, height);
tData.SetHeights(0, 0, GenerateHeights());
return tData;
}
float[,] GenerateHeights()
{
float[,] heights = new float[width, height];
for (int x = 0; x < width; x++)
{
for (int y = 0; y < height; y++)
{
//heights[x, y] = CalculateHeight(x, y);
var frames = pipe.WaitForFrames();
var depth = frames.DepthFrame;
heights[x, y] = depth.GetDistance(x, y);
}
}
return heights;
}
float CalculateHeight(int x, int y)
{
//float xCoord = (float)x / width * scale + xOffset;
//float yCoord = (float)y / height * scale + yOffset;
//return depthFrame.GetDistance(x,y);
//return Mathf.PerlinNoise(xCoord, yCoord);
using (var frames = pipe.WaitForFrames())
using (var depth = frames.DepthFrame)
return depth.GetDistance(x, y);
}
}
I am looking for some guidance on how to init the camera depth properly. I have never used a camera before with Unity.

Bit off-topic but I wanted to ask how well your above code runs? Are you getting errors such as:
ExternalException: rs2_pipeline_wait_for_frames(pipe:000000006402E3C0)
Rethrow as Exception: Frame didn't arrived within 1000
I'm trying to do something similar and also having problems. Maybe we can work it out together?
Cheers,
Cam

Related

Terrain having visible seams at vertices - How can i fix this?

i'm a newbie and i am following a tutorial on procedural landmass generation. However, my plane does not look right. It has a lot of seams/cracks. Is there someone who can point me in the right direction?
Below is my MeshGenerator scripts:
public static class MeshGenerator
{
public static MeshData GenerateTerrainMesh(float[,] heightMap, float heightMultiplier, AnimationCurve heightCurve)
{
int width = heightMap.GetLength(0);
int height = heightMap.GetLength(1);
float topLeftX = (width - 1) / -2f;
float topLeftZ = (height - 1) / 2f;
MeshData meshData = new MeshData(width, height);
int vertexIndex = 0;
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
meshData.vertices[vertexIndex] = new Vector3(topLeftX + x, heightCurve.Evaluate(heightMap[x,y]) * heightMultiplier, topLeftZ - y);
meshData.uvs[vertexIndex] = new Vector2(x / (float)width, y / (float)height);
if (x < width - 1 && y < height - 1)
{
meshData.AddTriangle(vertexIndex, vertexIndex + width + 1, vertexIndex + width);
meshData.AddTriangle(vertexIndex, + width + 1, vertexIndex + 1);
}
vertexIndex++;
}
}
return meshData;
}
}
public class MeshData
{
public Vector3[] vertices;
public int[] triangles;
public Vector2[] uvs;
int triangleIndex;
public MeshData(int meshWidth, int meshHeight)
{
vertices = new Vector3[meshWidth * meshHeight];
uvs = new Vector2[meshWidth * meshHeight];
triangles = new int[(meshWidth-1) * (meshHeight-1)*6];
}
public void AddTriangle(int a, int b, int c)
{
triangles[triangleIndex] = a;
triangles[triangleIndex+1] = b;
triangles[triangleIndex+2] = c;
triangleIndex += 3;
}
public Mesh CreateMesh()
{
Mesh mesh = new Mesh();
mesh.vertices = vertices;
mesh.triangles = triangles;
mesh.uv = uvs;
mesh.RecalculateNormals();
return mesh;
}
}
You triangle indices are wrong, this is rather obvious since you have a bunch of triangle-shaped holes. Notably
meshData.AddTriangle(vertexIndex, + width + 1, vertexIndex + 1);
the second vertex of the second triangle is a constant value, and that is most likely incorrect
You should not need to keep a running total of vertexIndex, you should be perfectly able to compute the triangle indices from the grid indices:
var v1 = y * (width+1) + x; // You should have one more column of vertices than you have grid cells
var v2 = v2 + 1; // the vertex one column to the right
var v3 = v1 + width+1; // the vertex one row down
var v4 = v3 + 1;
meshData.AddTriangle(v1, v2, v4);
meshData.AddTriangle(v1, v4, v3);
You may need to invert the vertex order to ensure the normals are oriented correctly.

Why cube mesh becomes a plane when in high resolution?

I use the following code to generate a Cube as a single mesh. My purpose is to generate a sphere from it by normalizing as I have shown in the commented line (I just have to do that to all those statements in the following lines). The problem here is that the mesh changes from a cube to a flat plane as I keep increasing the resolution (parameter given as public int resolution).
(This code was inspired by this video https://youtu.be/QN39W020LqU . But I am using the technique in my own way as given by the following code, so that I can generate a single mesh instead of a combination of 6 meshes, this is required for my work)
[code=CSharp]
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class Sc_Planet : MonoBehaviour
{
[Range(2, 512)]
public int resolution = 2;
[Range(2, 256)]
public int radius = 10;
MeshFilter meshFilter;
void OnValidate()
{
Initialize();
}
void Initialize()
{
if (meshFilter == null)
{
GameObject meshObj = new GameObject("mesh_Planet");
meshObj.transform.parent = transform;
meshObj.AddComponent<MeshRenderer>().sharedMaterial = new Material(Shader.Find("Standard"));
meshFilter = meshObj.AddComponent<MeshFilter>();
meshFilter.sharedMesh = new Mesh();
}
int xmax = resolution + 1;
int ymax = resolution + 1;
float dx = 1.0f / resolution;
float dy = 1.0f / resolution;
Vector3[] vertsTop = new Vector3[xmax * ymax];
Vector3[] vertsRight = new Vector3[xmax * ymax];
Vector3[] vertsFront = new Vector3[xmax * ymax];
Vector3[] vertsBottom = new Vector3[xmax * ymax];
Vector3[] vertsLeft = new Vector3[xmax * ymax];
Vector3[] vertsBack = new Vector3[xmax * ymax];
for (int y = 0; y < ymax; y++)
{
for (int x = 0; x < xmax; x++)
{
float px = dx * x - 0.5f;
float py = dy * y - 0.5f;
int t = x + y * xmax;
//vertsTop[t] = new Vector3(py, 0.5f, px).normalized * radius;
vertsTop[t] = new Vector3(py, 0.5f, px);
vertsRight[t] = new Vector3(px, py, 0.5f);
vertsFront[t] = new Vector3(0.5f, px, py);
vertsBottom[t] = new Vector3(px, -0.5f, py);
vertsLeft[t] = new Vector3(py, px, -0.5f);
vertsBack[t] = new Vector3(-0.5f, py, px);
}
}
List<int> trianglesList = new List<int>();
for (int y = 0; y < ymax - 1; ++y)
{
for (int x = 0; x < xmax; ++x)
{
if (x % xmax != xmax - 1)
{
int f = x + y * xmax;
trianglesList.Add(f);
trianglesList.Add(f + 1);
trianglesList.Add(f + 1 + xmax);
trianglesList.Add(f);
trianglesList.Add(f + 1 + xmax);
trianglesList.Add(f + xmax);
}
}
}
List<Vector3> verts = new List<Vector3>();
Dictionary<Vector3, int> vdict = new Dictionary<Vector3, int>();
List<int> triangles = new List<int>();
int nextIndex = 0;
void addFace(Vector3 [] in_verts, List<int> in_triangles)
{
for(int i = 0; i < in_verts.Length; ++i)
{
if (!vdict.ContainsKey(in_verts[i]))
{
vdict.Add(in_verts[i], nextIndex);
verts.Add(in_verts[i]);
++nextIndex;
}
}
for(int i = 0; i < in_triangles.Count; ++i)
{
triangles.Add(vdict[in_verts[in_triangles[i]]]);
}
}
addFace(vertsTop, trianglesList);
addFace(vertsRight, trianglesList);
addFace(vertsFront, trianglesList);
addFace(vertsBottom, trianglesList);
addFace(vertsLeft, trianglesList);
addFace(vertsBack, trianglesList);
var mesh = meshFilter.sharedMesh;
mesh.Clear();
mesh.vertices = verts.ToArray();
mesh.triangles = triangles.ToArray();
mesh.RecalculateNormals();
}
}
[/code]
This code works in Blender (I used python to script it on Blender and it works very well for any resolution).
The only problem is that when I use this in Unity, the meshes become weird as I have shown in the images I have attached below.
At Resolution = 96 :
At Resolution = 122 :
At Resolution = 182 :
At Resolution = 344:
Why is this happening?
How should I correct it?
(I have also posted this in unity forums: Why cube mesh becomes a plane when in high resolution?)
Ok I found the answer. This is exceeding the limit of vertices on unity api for 16-bit based meshes. I had to change it to a 32-bit indexed mesh to correct it.
Details are in this docuemntaiton page : https://docs.unity3d.com/ScriptReference/Rendering.IndexFormat.html?_ga=2.9556401.501737799.1635227368-67181881.1629608252
I just had to add the code :
mesh.indexFormat = UnityEngine.Rendering.IndexFormat.UInt32;
That was it.

Why does Unity get stuck on Application.EnterPlayMode?

I'm trying to create procedural terrain generator using perlin noise and marching cubes in unity. It was working until I switched from creating a height map to making it into a 3d array. Then, whenever I clicked on play, unity opened a dialog box with Application.EnterPlayMode written which would not go away and never entered play mode. Everything stops responding when it happens and they only way to stop it is to kill it in task manager.
The script in question is below:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class Noise : MonoBehaviour
{
//Determines whether to show debug values
public bool debug = false;
//Determines flatness of the terrain
public float noiseScale = 0.5f;
//Type of perlin noise to use
public enum PerlinNoise {
twoD,
threeD
};
public PerlinNoise perlinNoiseDimension = PerlinNoise.twoD;
//To return noise data after all calculations
public float[,,] getTerrainData(int x, int y, int z)
{
float[,,] terrainData = new float[x, y, z];
if(perlinNoiseDimension == PerlinNoise.twoD)
{
terrainData = PerlinNoise2D(x, y, z, noiseScale);
}
return terrainData;
}
//Determine noise values using 2D Perlin noise
private float[,,] PerlinNoise2D(int x, int y, int z, float noiseScale)
{
float[,,] voxelHeights = new float[x, y, z];
if (debug)
{
Debug.Log("Heightmap");
}
//Origin points to sample from
float xOrg = Random.Range(0.0f, 0.9999999f);
float yOrg = Random.Range(0.0f, 0.9999999f);
for (int currentx = 0; currentx < x; currentx++)
{
for (int currenty = 0; currenty < y; currenty++)
{
//Convert Values to Fractions
float xCoord = (float)currentx / (x * noiseScale) + xOrg;
float yCoord = (float)currenty / (y * noiseScale) + yOrg;
float height = Mathf.PerlinNoise(xCoord, yCoord) * z;
for(int currentz = 0; currentz <= height; z++)
{
voxelHeights[currentx, currenty, currentz] = 1;
}
if (debug)
{
Debug.Log("Height = " + height + ", X = " + currentx + ", Y = " + currenty + ", X Coord = " + xCoord + ", Y Coord = " + yCoord);
}
}
}
return voxelHeights;
}
}
An image of what it shows is below:
You are causing a never ending loop in
for(int currentz = 0; currentz <= height; z++)
{
voxelHeights[currentx, currenty, currentz] = 1;
}
here you are increasing z++ but your loop condition is on currentz <= height. Within the loop you never update the value of currentz nor the value of height so your loop never finishes.
Due to the usage of indices it should probably rather have been
for(int currentz = 0; currentz <= height; currentz++)
{
voxelHeights[currentx, currenty, currentz] = 1;
}
However not sure how the height comes into play here because I would expect it rather to look somewhat like e.g.
for(int currentz = 0; currentz < z; currentz++)
{
voxelHeights[currentx, currenty, currentz] = height;
}
That seems to make more sense to me.

How to modify perlin noise to generate a simple shape?

I am working on a project in Unity that needs to generate some simple 3D Mountains/Hills. Since my requirements are to create a "simple" shape, I don't seem to find an answer and I thought maybe I can get some help from here. Anyways, this is a normal output from perlin noise, although it's smooth, the output is still complicated with lots of hills/mountains. I am looking for something like this . I need to be sure that I won't have any height around the borders of the Output Image. I think you've got the idea. Have a Great Day!
Here is the code I am using right now from an online tutorial:
using UnityEngine;
public class PerlinNoise : MonoBehaviour
{
private static int width = 256;
private static int height = 128;
public float scale = 20f;
public float offsetX = 100f;
public float offsetY = 100f;
private int xcont = 0, ycont = 0;
public float[,] array = new float[width,height];
private void Start()
{
offsetX = Random.Range(0f, 99999f);
offsetY = Random.Range(0f, 99999f);
}
void Update()
{
Renderer renderer = GetComponent<Renderer>();
renderer.material.mainTexture = GenerateTexture();
}
Texture2D GenerateTexture()
{
Texture2D texture = new Texture2D(width, height);
//GENERATE A PERLIN NOISE MAP FOR THE TEXTURE
for(int x=0;x<width;x++)
{
for(int y=0;y<height;y++)
{
Color color = CalculateColor(x,y);
texture.SetPixel(x, y, color);
}
}
texture.Apply();
return texture;
}
Color CalculateColor(int x, int y)
{
float xCoord = (float)x / width * scale + offsetX;
float yCoord = (float)y / height * scale + offsetY;
float sample = Mathf.PerlinNoise(xCoord,yCoord);
if (xcont == width - 1)
{
xcont = 0;
ycont++;
}
else xcont++;
if (ycont == height - 1 ) ycont = 0;
array[xcont,ycont] = sample;
return new Color(sample, sample, sample);
}
}
You can filter that texture down using cellular automata.
This playlist might help you understand how and when Perlin Noise is used for map generation: https://www.youtube.com/playlist?list=PLFt_AvWsXl0eZgMK_DT5_biRkWXftAOf9

How to determine scale to increase sprite width to screen width

I am trying to achieve something similar to How to increase (animate) the width of the square on both ends in Unity. How can I determine the scale by which to increase the width (of the sprite) for it to fill the whole screen width?
UPDATE
Below is the Swift code for I implemented for expanding the sprite width to take the full screen width:
func expandEnemy () {
spritePosBeforeScaleX = CGPointMake((enemy?.sprite.position.x)!, (enemy?.sprite.anchorPoint.y)!)
enemy?.sprite.anchorPoint = CGPointMake((enemy?.sprite.position.x)! / self.size.width, (enemy?.sprite.anchorPoint.y)!)
let enemyScalingAction = SKAction.scaleXTo(self.size.width / (enemy?.sprite.size.width)!, duration: 1.0)
enemy!.sprite.runAction(enemyScalingAction)
delay(0.1)
{
center = CGPointMake(enemy!.sprite.size.width / 2 - (enemy!.sprite.size.width * enemy!.sprite.anchorPoint.x), enemy!.sprite.size.height / 2 - (enemy!.sprite.size.height * enemy!.sprite.anchorPoint.y))
enemy!.sprite.physicsBody = SKPhysicsBody(rectangleOfSize: enemy!.sprite.size, center: center)
}
}
It all depends on the aspect ratio of the screen and the size of the object with the SpriteRenderer. You need to scale up the gameobject that holds the spriterenderer by a factor where you take these into consideration.
[ExecuteInEditMode]
public class SpriteToScreen : MonoBehaviour {
public float sprw = 256f;
public float sprh = 256f;
float unitspp = 100f;
public float scrw = 0f;
public float scrh = 0f;
public float aspect = 0f;
public float spr_aspect = 1f;
public float factorY = 0.017578125f;
public void Update(){
scrw = Screen.width;
scrh = Screen.height;
aspect = scrw / scrh;
unitspp = this.GetComponent<SpriteRenderer>().sprite.pixelsPerUnit;
sprw = this.GetComponent<SpriteRenderer>().sprite.bounds.size.x * unitspp;
sprh = this.GetComponent<SpriteRenderer>().sprite.bounds.size.y * unitspp;
spr_aspect = sprw / sprh;
this.transform.localScale = new Vector3( (1152f / sprh * aspect) / spr_aspect,
1152f / sprh,
1 );
}
}
You can scale Image of an UI in x-axis full screen. Just get the RectTransform then modify the sizeDelta property of it to the Screen size of the device or the size of the Canvas.
The function below can scale Unity UI Image in x, y or both x and y axis full screen. The Image to scale must be under Canvas. Assign a Sprite to the Source Image of the Image component the code below should work.
//Attach the UI Image to scacle in the Editor here
public GameObject image;
To Scale:
Scale in X-axis Full Screen in 3 seconds:
StartCoroutine(scaleToFullScreen(image, 0, 3f));
Scale in Y-axis Full Screen in 3 seconds:
StartCoroutine(scaleToFullScreen(image, 1, 3f));
Scale in X-axis AND Y-axis Full Screen in 3 seconds:
StartCoroutine(scaleToFullScreen(image, 2, 3f));
The Scale function:
bool isScaling = false;
IEnumerator scaleToFullScreen(GameObject imageToScale, int scaleType, float byTime)
{
if (isScaling)
{
yield break;
}
if (scaleType < 0 || scaleType > 2)
{
Debug.Log("Invalid ScaleType. Valid Scale Types X:0, Y:1, XandY:3");
yield break;
}
isScaling = true;
Canvas canvas = imageToScale.GetComponentInParent<Canvas>();
float x, y;
if (canvas != null)
{
x = canvas.pixelRect.width;
y = canvas.pixelRect.height;
}
else
{
x = Screen.width;
y = Screen.height;
}
RectTransform rect = imageToScale.GetComponent<RectTransform>();
if (rect == null)
{
rect = imageToScale.AddComponent<RectTransform>();
}
//Center the position of the image so that it will be resized equally
rect.anchoredPosition3D = new Vector3(0, 0, rect.anchoredPosition3D.z);
//The default Size
Vector2 originalScale = rect.sizeDelta;
//The new scale we want to scale texture to
Vector2 newScale = originalScale;
if (scaleType == 0)
{
newScale.x = x;
}
else if (scaleType == 1)
{
newScale.y = y;
}
else if (scaleType == 2)
{
newScale.x = x;
newScale.y = y;
}
float counter = 0;
while (counter < byTime)
{
counter += Time.deltaTime;
rect.sizeDelta = Vector2.Lerp(originalScale, newScale, counter / byTime);
yield return null;
}
isScaling = false;
}

Categories

Resources