Develop Reference

Facial detection coordinates using a camera

I need a way to grab the coordinates of the face in C# for Windows Phone 8.1 in the camera view. I haven't been able to find anything on the web so I'm thinking it might not be possible. What I need is the x and y (and possibly area) of the "box" that forms around the face when it is detected in the camera view. Has anyone done this before?

Code snippet (bear in mind this is part of an app from the tutorial I linked below the code. It's not copy-pasteable, but should provide some help)
const string MODEL_FILE = "haarcascade_frontalface_alt.xml";
FaceDetectionWinPhone.Detector m_detector;
public MainPage()
{
InitializeComponent();
m_detector = new FaceDetectionWinPhone.Detector(System.Xml.Linq.XDocument.Load(MODEL_FILE));
}
void photoChooserTask_Completed(object sender, PhotoResult e)
{
if (e.TaskResult == TaskResult.OK)
{
BitmapImage bmp = new BitmapImage();
bmp.SetSource(e.ChosenPhoto);
WriteableBitmap btmMap = new WriteableBitmap(bmp);
//find faces from the image
List<FaceDetectionWinPhone.Rectangle> faces =
m_detector.getFaces(
btmMap, 10f, 1f, 0.05f, 1, false, false);
//go through each face, and draw a red rectangle on top of it.
foreach (var r in faces)
{
int x = Convert.ToInt32(r.X);
int y = Convert.ToInt32(r.Y);
int width = Convert.ToInt32(r.Width);
int height = Convert.ToInt32(r.Height);
btmMap.FillRectangle(x, y, x + height, y + width, System.Windows.Media.Colors.Red);
}
//update the bitmap before drawing it.
btmMap.Invalidate();
facesPic.Source = btmMap;
}
}
This is taken from developer.nokia.com
To do this in real-time, you need to intercept the viewfinder image, perhaps using the NewCameraFrame method (EDIT: not sure if you should use this method or PhotoCamera.GetPreviewBufferArgb32 as described below. I have to leave it up to your research)
So basically your task has 2 parts:
Get the viewfinder image
Detect faces on it (using something like the code above)
If I were you, I'd first do step 2. on an image loaded from disk, and once you can detect faces on that, I'd see how to obtain current viewfinder image and detect faces on that. X,Y coordinates are easy enough to obtain once you've detected the face - see code above.
(EDIT): I think you should try using PhotoCamera.GetPreviewBufferArgb32 method to obtain the viewfinder image. Look here MSDN documentation. Also, be sure to search through MSDN docs and tutorials. This should be more than enough to complete step 1.
A lot of face detection algorithms use Haar classifiers, Viola-Jones algorithm etc. If you're familiar with that, you'll feel more confident in what you're doing, but you can do without. Also, read the materials that I linked - they seem fairly good.

Changing Sprite and keeping the same size

I'm writing a game using the 2d features of unity.
I'm designing a sort of inventory for the player character, and I have a gameobject with a number of placeholder images inside it. The intention is that when I actually load this gameobject, I'll replace the sprites of the placeholder images and I'll display what I want.
My issue is that when I change the sprite using code like this
var ren = item1.GetComponentInChildren<SpriteRenderer>();
ren.sprite = Resources.Load<Sprite>("DifferentSprite");
The image loaded is correct, however the scaling applies to the new sprite. The issue is that these sprites have all different sizes. So whilst the original placeholder image takes up a small square, the replacement might be tiny or massive enough to cover the whole screen depending on how the actual png was sized.
I basically want the sprite to replace the other and scale itself such that it has the same width and height as the placeholder image did. How can I do this?
EDIT - I've tried playing around with ratios. It's still not working perfectly, but its close.
var ren = item1.GetComponentInChildren<SpriteRenderer>();
Vector2 beforeSize = ren.transform.renderer.bounds.size;
ren.sprite = Resources.Load<Sprite>("Day0/LampOn");
Vector2 spriteSize = ren.sprite.bounds.size;
//Get the ratio between the wanted size and the sprite's size ratios
Vector3 scaleMultiplier = new Vector3 (beforeSize.x/spriteSize.x, beforeSize.y/spriteSize.y);
//Multiple the scale by this multiplier
ren.transform.localScale = new Vector3(ren.transform.localScale.x * scaleMultiplier.x,ren.transform.localScale.y * scaleMultiplier.y);

puzzle.Sprite = Sprite.Create(t, new Rect(0, 0, t.width, t.height),Vector2.one/2,256);
the last int is for the pixelperUnity, its the cause of changing size.
It's fixed in my project,
my default pixelPerUnit=256 and unity default = 100 so its bigger 2.56 times
tell me if it helps

How to modify width and height: use RectTransform.sizeDelta.
Example for this type of script
Vector3 originalDelta = imageToSwap.rectTransform.sizeDelta; //get the original Image's rectTransform's size delta and store it in a Vector called originalDelta
imageToSwap.sprite = newSprite; //swap out the new image
imageToSwap.rectTransform.sizeDelta = originalDelta; //size it as the old one was.
More about sizeDeltas here

I would expose two public variables in the script, with some default values:
public int placeholderWidth = 80;
public int placeholderHeight = 80;
Then, whenever you change sprites, set the width & height to those pre-defined values.

XNA Hardware Instancing: Mesh not rendered completely

I have implemented basic Hardware model instancing method in XNA code by following this short tutorial:
http://www.float4x4.net/index.php/2011/07/hardware-instancing-for-pc-in-xna-4-with-textures/
I have created the needed shader (without texture atlas though, single texture only) and I am trying to use this method to draw a simple tree I generated using 3DS Max 2013 and exported via FBX format.
The results I'm seeing left me without clue as to what is going on.
Back when I was using no instancing methods, but simply calling Draw on a mesh (for every tree on a level), the whole tree was shown:
I have made absolutely sure that the Model contains only one Mesh and that Mesh contains only one MeshPart.
I am using Vertex Extraction method, by using Model's Vertex and Index Buffer "GetData<>()" method, and correct number of vertices and indices, hence, correct number of primitives is rendered. Correct texture coordinates and Normals for lighting are also extracted, as is visible by the part of the tree that is being rendered.
Also the parts of the tree are also in their correct places as well.
They are simply missing some 1000 or so polygons for absolutely no reason what so ever. I have break-pointed at every step of vertex extraction and shader's parameter generation, and I cannot for the life of me figure out what am I doing wrong.
My Shader's Vertex Transformation function:
VertexShaderOutput VertexShaderFunction2(VertexShaderInput IN, float4x4 instanceTransform : TEXCOORD1)
{
VertexShaderOutput output;
float4 worldPosition = mul(IN.Position, transpose(instanceTransform));
float4 viewPosition = mul(worldPosition, View);
output.Position = mul(viewPosition, Projection);
output.texCoord = IN.texCoord;
output.Normal = IN.Normal;
return output;
}
Vertex bindings and index buffer generation:
instanceBuffer = new VertexBuffer(Game1.graphics.GraphicsDevice, Core.VertexData.InstanceVertex.vertexDeclaration, counter, BufferUsage.WriteOnly);
instanceVertices = new Core.VertexData.InstanceVertex[counter];
for (int i = 0; i < counter; i++)
{
instanceVertices[i] = new Core.VertexData.InstanceVertex(locations[i]);
}
instanceBuffer.SetData(instanceVertices);
bufferBinding[0] = new VertexBufferBinding(vBuffer, 0, 0);
bufferBinding[1] = new VertexBufferBinding(instanceBuffer, 0, 1);
Vertex extraction method used to get all vertex info (this part I'm sure works correctly as I have used it before to load test geometric shapes into levels, like boxes, spheres, etc for testing various shaders, and constructing bounding boxes around them using extracted vertex data, and it is all correct):
public void getVertexData(ModelMeshPart part)
{
modelVertices = new VertexPositionNormalTexture[part.NumVertices];
rawData = new Vector3[modelVertices.Length];
modelIndices32 = new uint[rawData.Length];
modelIndices16 = new ushort[rawData.Length];
int stride = part.VertexBuffer.VertexDeclaration.VertexStride;
VertexPositionNormalTexture[] vertexData = new VertexPositionNormalTexture[part.NumVertices];
part.VertexBuffer.GetData(part.VertexOffset * stride, vertexData, 0, part.NumVertices, stride);
if (part.IndexBuffer.IndexElementSize == IndexElementSize.ThirtyTwoBits)
part.IndexBuffer.GetData<uint>(modelIndices32);
if (part.IndexBuffer.IndexElementSize == IndexElementSize.SixteenBits)
part.IndexBuffer.GetData<ushort>(modelIndices16);
for (int i = 0; i < modelVertices.Length; i++)
{
rawData[i] = vertexData[i].Position;
modelVertices[i].Position = rawData[i];
modelVertices[i].TextureCoordinate = vertexData[i].TextureCoordinate;
modelVertices[i].Normal = vertexData[i].Normal;
counter++;
}
}
This is the rendering code for the object batch (trees in this particular case):
public void RenderHW()
{
Game1.graphics.GraphicsDevice.RasterizerState = rState;
treeBatchShader.CurrentTechnique.Passes[0].Apply();
Game1.graphics.GraphicsDevice.SetVertexBuffers(bufferBinding);
Game1.graphics.GraphicsDevice.Indices = iBuffer;
Game1.graphics.GraphicsDevice.DrawInstancedPrimitives(PrimitiveType.TriangleList, 0, 0, treeMesh.Length, 0, primitive , counter);
Game1.graphics.GraphicsDevice.RasterizerState = rState2;
}
If anybody has any idea where to even start looking for errors, just post all ideas that come to mind, as I'm completely stumped as to what's going on.
This even counters all my previous experience where I'd mess something up in shader code or vertex generation, you'd get some absolute mess on your screen - numerous graphical artifacts such as elongated triangles originating where mesh should be, but one tip stretching back to (0,0,0), black texture, incorrect positioning (often outside skybox or below terrain), incorrect scaling...
This is something different, almost as if it works - the part of the tree that is visible is correct in every single aspect (location, rotation, scale, texture, shading), except that a part is missing. What makes it weirder for me is that the part missing is seemingly logically segmented: Only tree trunk's primitives, and some leaves off the lowest branches of the tree are missing, leaving all other primitives correctly rendered with no artifacts. Basically, they're... correctly missing.

Solved. Of course it was the one part I was 100% sure it was correct while it was not.
modelIndices32 = new uint[rawData.Length];
modelIndices16 = new ushort[rawData.Length];
Change that into:
modelIndices32 = new uint[part.IndexBuffer.IndexCount];
modelIndices16 = new ushort[part.IndexBuffer.IndexCount];
Now I have to just figure out why are 3 draw calls rendering 300 trees slower than 300 draw calls rendering 1 tree each (i.e. why did I waste entire afternoon creating a new problem).

Applying vtkWarpLens to Camera vs. Scene Data

Can anyone help me with vtkWarpLens?
What I am trying to do is implement a distortion pattern on the camera to modify how the data is seen.
Here is the meat of the code (it's in C#, I'm using Activis)
vtkPolyData pd = vtkPolyData.New();
CreateFromFile(pd); // this creates a triangle representation of a height field
double [] sr = pd.GetScalarRange();
vtkLookupTable lut = vtkLookupTable.New();
lut.SetNumberOfColors(16);
lut.SetHueRange(0.667, 0.0);
lut.Build();
wl = vtkWarpLens.New();
wl.SetInputConnection(pd.GetProducerPort());
wl.SetPrincipalPoint(0.5, 0.5);
wl.SetFormatWidth(1);
wl.SetFormatHeight(1);
wl.SetImageWidth(1000);
wl.SetImageHeight(1000);
wl.SetK1(0.01307);
wl.SetK2(0.0003102);
wl.SetP1(1.953e-005);
wl.SetP2(-9.655e-005);
vtkDataSetMapper dsmDistorted = vtkDataSetMapper.New();
dsmDistorted.SetInputConnection(wl.GetOutputPort());
dsmDistorted.SetLookupTable(lut);
dsmDistorted.SetScalarRange(sr[0]+20, sr[1]);
vtkActor dsDistortedActor = vtkActor.New();
dsDistortedActor.SetMapper(dsmDistorted);
m_renDistorted.AddActor(dsDistortedActor);
m_renWin.SetDesiredUpdateRate(0);
m_renWin.Render();
m_renDistorted.ResetCamera();
So, basically, I am creating a terrain representation using polygons, passing it through the warplens, passing that through a dataset mapper to give it pretty colors, then displaying it.
The issue is that the "warp" appears to be static to the terrain, not to the camera. I'm pretty new to VTK, so it's possible that I don't understand how the Interactor and the Camera are related.
Can someone help?

How do you do a 3D transform (perspective) in C# or VB.Net?

What I am looking to do sounds really simple, but no where on the Internet so far have I found a way to do this in DotNet nor found a 3rd party component that does this either (without spending thousands on completely unnecessary features).
Here goes:
I have a jpeg of a floor tile (actual photo) that I create a checkerboard pattern with.
In dotnet, it is easy to rotate and stitch photos together and save the final image as a jpeg.
Next, I want to take that final picture and make it appear as if the "tiles" are laying on a floor for a generic "room scene". Basically adding a 3D perspective to make it appear as if it is actually in the room scene.
Heres a website that is doing something similar with carpeting, however I need to do this in a WinForms application:
Flor Website
Basically, I need to create a 3D perspective of a jpeg, then save it as a new jpeg (then I can put an overlay of the generic room scene).
Anyone have any idea on where to get a 3rd party DotNet image processing module that can do this seemingly simple task?

It is not so simple because you need a 3D transformation, which is more complicated and computationally expensive than a simple 2D transformation such as rotation, scaling or shearing. For you to have an idea of the difference in the math, 2D transformations require 2 by 2 matrices, whereas a projection transformation (which is more complicated than other 3D transforms) requires a 4 by 4 matrix...
What you need is some 3D rendering engine in which you can draw polygons (in a perspective view) and them cover them with a texture (like a carpet). For .Net 2.0, I'd recommend using SlimDX which is a port of DirectX that would allow you to render polygons, but there is some learning curve. If you are using WPF (.Net 3.0 and up), there is a built in 3D canvas that allows you to draw textured polygons in perspective. That might be easier/better to learn than SlimDX for your purposes. I'm sure that there is a way to redirect the output of the 3D canvas towards a jpeg...
You might simplify the problem a lot if you don't require great performance and if you restrict the orientation of the texture (eg. always a horizontal floor or always a vertical wall). If so, you could probably render it yourself with a simple drawing loop in .Net 2.0.

If you just want a plain floor, your code would look like this. WARNING: Obtaining your desired results will take some significant time and refinement, specially if you don't know the math very well. But on the other hand, it is always fun to play with code of this type... (:
Find some sample images below.
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Windows.Forms;
namespace floorDrawer
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
ResizeRedraw = DoubleBuffered = true;
Width = 800;
Height = 600;
Paint += new PaintEventHandler(Form1_Paint);
}
void Form1_Paint(object sender, PaintEventArgs e)
{
// a few parameters that control the projection transform
// these are the parameters that you can modify to change
// the output
double cz = 10; // distortion
double m = 1000; // magnification, usually around 1000 (the pixel width of the monitor)
double y0 = -100; // floor height
string texturePath = #"c:\pj\Hydrangeas.jpg";//#"c:\pj\Chrysanthemum.jpg";
// screen size
int height = ClientSize.Height;
int width = ClientSize.Width;
// center of screen
double cx = width / 2;
double cy = height / 2;
// render destination
var dst = new Bitmap(width, height);
// source texture
var src = Bitmap.FromFile(texturePath) as Bitmap;
// texture dimensions
int tw = src.Width;
int th = src.Height;
for (int y = 0; y < height; y++)
for (int x = 0; x < width; x++)
{
double v = m * y0 / (y - cy) - cz;
double u = (x - cx) * (v + cz) / m;
int uu = ((int)u % tw + tw) % tw;
int vv = ((int)v % th + th) % th;
// The following .SetPixel() and .GetPixel() are painfully slow
// You can replace this whole loop with an equivalent implementation
// using pointers inside unsafe{} code to make it much faster.
// Note that by casting u and v into integers, we are performing
// a nearest pixel interpolation... It's sloppy but effective.
dst.SetPixel(x, y, src.GetPixel(uu, vv));
}
// draw result on the form
e.Graphics.DrawImage(dst, 0, 0);
}
}
}