Develop Reference

OpenCV FindContours does not find the correct rectangle

Using CvInvoke.Canny and CvInvoke.FindContours I'm trying to find the rectangle containing the item name (Schematic: Maple Desk). This rectangle is shown in the image below:
I'm able to find a lot of rectangles but I'm not able to get this one. Tried a lot of different thresholds for Canny but to no effect. The following image shows all rectangles I currently get:
Any ideas how to tackle this? Do I need to use other thresholds or another approach? I already experimented using grayscale and blurring but that didn't give better result. I added my current source below and the original image I'm using is this:
public Mat ProcessImage(Mat img)
{
UMat filter = new UMat();
UMat cannyEdges = new UMat();
Mat rectangleImage = new Mat(img.Size, DepthType.Cv8U, 3);
//Convert the image to grayscale and filter out the noise
//CvInvoke.CvtColor(img, filter, ColorConversion.Bgr2Gray);
//Remove noise
//CvInvoke.GaussianBlur(filter, filter, new System.Drawing.Size(3, 3), 1);
// Canny and edge detection
double cannyThreshold = 1.0; //180.0
double cannyThresholdLinking = 1.0; //120.0
//CvInvoke.Canny(filter, cannyEdges, cannyThreshold, cannyThresholdLinking);
CvInvoke.Canny(img, cannyEdges, cannyThreshold, cannyThresholdLinking);
// Find rectangles
List<RotatedRect> rectangleList = new List<RotatedRect>();
using (VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint())
{
CvInvoke.FindContours(cannyEdges, contours, null, RetrType.List, ChainApproxMethod.ChainApproxSimple);
int count = contours.Size;
for (int i = 0; i < count; i++)
{
using (VectorOfPoint contour = contours[i])
using (VectorOfPoint approxContour = new VectorOfPoint())
{
CvInvoke.ApproxPolyDP(contour, approxContour, CvInvoke.ArcLength(contour, true) * 0.05, true);
// Only consider contours with area greater than 250
if (CvInvoke.ContourArea(approxContour, false) > 250)
{
// The contour has 4 vertices.
if (approxContour.Size == 4)
{
// Determine if all the angles in the contour are within [80, 100] degree
bool isRectangle = true;
System.Drawing.Point[] pts = approxContour.ToArray();
LineSegment2D[] edges = Emgu.CV.PointCollection.PolyLine(pts, true);
for (int j = 0; j < edges.Length; j++)
{
double angle = Math.Abs(edges[(j + 1) % edges.Length].GetExteriorAngleDegree(edges[j]));
if (angle < 80 || angle > 100)
{
isRectangle = false;
break;
}
}
if (isRectangle) rectangleList.Add(CvInvoke.MinAreaRect(approxContour));
}
}
}
}
}
// Draw rectangles
foreach (RotatedRect rectangle in rectangleList)
{
CvInvoke.Polylines(rectangleImage, Array.ConvertAll(rectangle.GetVertices(), System.Drawing.Point.Round), true,
new Bgr(Color.DarkOrange).MCvScalar, 2);
}
//Drawing a light gray frame around the image
CvInvoke.Rectangle(rectangleImage,
new Rectangle(System.Drawing.Point.Empty,
new System.Drawing.Size(rectangleImage.Width - 1, rectangleImage.Height - 1)),
new MCvScalar(120, 120, 120));
//Draw the labels
CvInvoke.PutText(rectangleImage, "Rectangles", new System.Drawing.Point(20, 20),
FontFace.HersheyDuplex, 0.5, new MCvScalar(120, 120, 120));
Mat result = new Mat();
CvInvoke.VConcat(new Mat[] { img, rectangleImage }, result);
return result;
}
Edit 1:
After some more fine tuning with the following thresholds for Canny
cannyThreshold 100
cannyThresholdLinking 400
And using a minimum size for all ContourAreas of 10000 I can get the following result:
Edit 2:
For those interested, solved using the current detection, no changes were needed. Used the 2 detected rectangles in the screenshot above to get the location of the missing rectangle containing the item name.
Result can be found here:
https://github.com/josdemmers/NewWorldCompanion

For those interested, solved using the current detection, no changes were needed. Used the 2 detected rectangles in the screenshot above to get the location of the missing rectangle containing the item name.
Result can be found here: https://github.com/josdemmers/NewWorldCompanion

Calculate area of polygon having WGS coordinates using DotSpatial?

Calculating area:
var coordinates = new List<Coordinate> {
new Coordinate(55, 35),
new Coordinate(55, 35.1),
new Coordinate(55.1, 35.1),
new Coordinate(55.1, 35),
};
Console.WriteLine(new Polygon(coordinates).Area); // ~0.01
Calculation is right, because it's happen in orthogonal coordinate system.
But how to mark that coordinates are in WGS?

It seems that task is more complicated that I've expected. I found this useful discussion on google groups
Firstly we need to found projection system, that is most suitable for our region where we need to compute area. For example you can take one of UTM zones
using DotSpatial.Projections;
using DotSpatial.Topology;
public static double CalculateArea(IEnumerable<double> latLonPoints)
{
// source projection is WGS1984
var projFrom = KnownCoordinateSystems.Geographic.World.WGS1984;
// most complicated problem - you have to find most suitable projection
var projTo = KnownCoordinateSystems.Projected.UtmWgs1984.WGS1984UTMZone37N;
// prepare for ReprojectPoints (it mutates array)
var z = new double[latLonPoints.Count() / 2];
var pointsArray = latLonPoints.ToArray();
Reproject.ReprojectPoints(pointsArray, z, projFrom, projTo, 0, pointsArray.Length / 2);
// assemblying new points array to create polygon
var points = new List<Coordinate>(pointsArray.Length / 2);
for (int i = 0; i < pointsArray.Length / 2; i++)
points.Add(new Coordinate(pointsArray[i * 2], pointsArray[i * 2 + 1]));
var poly = new Polygon(points);
return poly.Area;
}

You can get the area directly from IGeometry or from Feature.Geometry. Also You need to repeat the first coordinate to close your polygon.
FeatureSet fs = new FeatureSet(FeatureType.Polygon);
Coordinate[] coord = new Coordinate[]
{
new Coordinate(55, 35),
new Coordinate(55, 35.1),
new Coordinate(55.1, 35.1),
new Coordinate(55.1, 35),
new Coordinate(55, 35)
};
fs.AddFeature(new Polygon(new LinearRing(coord)));
var area = fs.Features.First().Geometry.Area;

Reading Datastream sharpDX Error all values are 0

I followed this solution for my project : How to create bitmap from Surface (SharpDX)
I don't have enough reputation to comment so I'm opening a new question here.
My project is basically in Direct 2D, I have a Surface buffer, a swapchain. I want to put my buffer into a datastream and reads it's value to put it into a bitmap and save it on disk ( like a screen capture), but my code won't work since all the bytes values are 0 (which is black) and this doesn't make sense since my image is fully white with a bit of blue.
Here is my code :
SwapChainDescription description = new SwapChainDescription()
{
ModeDescription = new ModeDescription(this.Width, this.Height, new Rational(60, 1), Format.B8G8R8A8_UNorm),
SampleDescription = new SampleDescription(1, 0),
Usage = Usage.RenderTargetOutput,
BufferCount = 1,
SwapEffect = SwapEffect.Discard,
IsWindowed = true,
OutputHandle = this.Handle
};
Device.CreateWithSwapChain(DriverType.Hardware, DeviceCreationFlags.Debug | DeviceCreationFlags.BgraSupport, description, out device, out swapChain);
SharpDX.DXGI.Device dxgiDevice = device.QueryInterface<SharpDX.DXGI.Device>();
SharpDX.DXGI.Adapter dxgiAdapter = dxgiDevice.Adapter;
SharpDX.Direct2D1.Device d2dDevice = new SharpDX.Direct2D1.Device(dxgiDevice);
d2dContext = new SharpDX.Direct2D1.DeviceContext(d2dDevice, SharpDX.Direct2D1.DeviceContextOptions.None);
SharpDX.Direct3D11.DeviceContext d3DeviceContext = new SharpDX.Direct3D11.DeviceContext(device);
properties = new BitmapProperties(new SharpDX.Direct2D1.PixelFormat(SharpDX.DXGI.Format.B8G8R8A8_UNorm, SharpDX.Direct2D1.AlphaMode.Premultiplied),
96, 96);
Surface backBuffer = swapChain.GetBackBuffer<Surface>(0);
d2dTarget = new SharpDX.Direct2D1.Bitmap(d2dContext, backBuffer, properties);
d2dContext.Target = d2dTarget;
playerBitmap = this.LoadBitmapFromContentFile(#"C:\Users\ndesjardins\Desktop\wave.png");
//System.Drawing.Bitmap bitmapCanva = new System.Drawing.Bitmap(1254, 735);
d2dContext.BeginDraw();
d2dContext.Clear(SharpDX.Color.White);
d2dContext.DrawBitmap(playerBitmap, new SharpDX.RectangleF(0, 0, playerBitmap.Size.Width, playerBitmap.Size.Height), 1f, SharpDX.Direct2D1.BitmapInterpolationMode.NearestNeighbor);
SharpDX.Direct2D1.SolidColorBrush brush = new SharpDX.Direct2D1.SolidColorBrush(d2dContext, SharpDX.Color.Green);
d2dContext.DrawRectangle(new SharpDX.RectangleF(200, 200, 100, 100), brush);
d2dContext.EndDraw();
swapChain.Present(1, PresentFlags.None);
Texture2D backBuffer3D = backBuffer.QueryInterface<SharpDX.Direct3D11.Texture2D>();
Texture2DDescription desc = backBuffer3D.Description;
desc.CpuAccessFlags = CpuAccessFlags.Read;
desc.Usage = ResourceUsage.Staging;
desc.OptionFlags = ResourceOptionFlags.None;
desc.BindFlags = BindFlags.None;
var texture = new Texture2D(device, desc);
d3DeviceContext.CopyResource(backBuffer3D, texture);
byte[] data = null;
using (Surface surface = texture.QueryInterface<Surface>())
{
DataStream dataStream;
var map = surface.Map(SharpDX.DXGI.MapFlags.Read, out dataStream);
int lines = (int)(dataStream.Length / map.Pitch);
data = new byte[surface.Description.Width * surface.Description.Height * 4];
dataStream.Position = 0;
int dataCounter = 0;
// width of the surface - 4 bytes per pixel.
int actualWidth = surface.Description.Width * 4;
for (int y = 0; y < lines; y++)
{
for (int x = 0; x < map.Pitch; x++)
{
if (x < actualWidth)
{
data[dataCounter++] = dataStream.Read<byte>();
}
else
{
dataStream.Read<byte>();
}
}
}
dataStream.Dispose();
surface.Unmap();
int width = surface.Description.Width;
int height = surface.Description.Height;
byte[] bytewidth = BitConverter.GetBytes(width);
byte[] byteheight = BitConverter.GetBytes(height);
Array.Copy(bytewidth, 0, data, 0, 4);
Array.Copy(byteheight, 0, data, 4, 4);
}
Do you guys have any idea why the byte array that is returned at the end is full of 0 since it should be mostly 255? All I did in my backbuffer was to draw a bitmap image and a rectangle form. Array.Copy is to add the width and height header to the byte array, therefore I could create a bitmap out of it.

I answered in a comment but formatting is horrible so apologies!
https://gamedev.stackexchange.com/a/112978/29920 This looks promising but as you said in reply to mine, this was some time ago and I'm pulling this out of thin air, if it doesn't work either someone with more current knowledge will have to answer or I'll have to grab some source code and try myself.
SharpDX.Direct2D1.Bitmap dxbmp = new SharpDX.Direct2D1.Bitmap(renderTarget,
new SharpDX.Size2(bmpWidth, bmpHeight), new
BitmapProperties(renderTarget.PixelFormat));
dxbmp.CopyFromMemory(bmpBits, bmpWidth * 4);
This looks kind of like what you need. I'm assuming bmpBits in here is either a byte array or a memory stream either of which could then be saved off or at least give you something to look at to see if you're actually getting pixel data

Find Black Squares/Rectangles in Image with C#

I want to find small black squares/rectangles in scanned sheet to:
deskewing image if needed.
remove white page margin.
Input image:
sample image 1 http://us.cdn.persiangig.com/preview/ii2jf6/2.jpg
Output image:
sample image 2 http://us.cdn.persiangig.com/preview/9ntpnc/1.jpg
My code to find square is:
Bitmap pic =(Bitmap) pictureBox1.Image;
// create filter
AForge.Imaging.Filters.Median Medianfilter = new AForge.Imaging.Filters.Median();
// apply the filter
Medianfilter.ApplyInPlace(pic);
Bitmap grayImage;
if (pic.PixelFormat != PixelFormat.Format16bppGrayScale && pic.PixelFormat != PixelFormat.Format8bppIndexed)
{
AForge.Imaging.Filters.Grayscale grayscalefilter = new AForge.Imaging.Filters.Grayscale(0.2125, 0.7154, 0.0721);
grayImage = grayscalefilter.Apply((Bitmap)pictureBox1.Image);
}
else
{
grayImage = pic;
}
// black & white:
Threshold(ref grayImage, Convert.ToInt32(textBox1.Text), Convert.ToInt32(textBox1.Text));
// invert filter
Invert invertfilter = new Invert();
// apply the filter
invertfilter.ApplyInPlace(grayImage);
// Edge Detector filter
DifferenceEdgeDetector EdgeDetectorfilter = new DifferenceEdgeDetector();
// apply the filter
EdgeDetectorfilter.ApplyInPlace(grayImage);
// create filter
Dilatation Dilatationfilter = new Dilatation();
// apply the filter
Dilatationfilter.ApplyInPlace(grayImage);
Finding object(square/rectangle) in image:
// lock image
BitmapData bitmapData = grayImage.LockBits(new Rectangle(0, 0, grayImage.Width, grayImage.Height),
ImageLockMode.ReadWrite, grayImage.PixelFormat);
// step 2 - locating objects
BlobCounter blobCounter = new BlobCounter();
blobCounter.FilterBlobs = true;
blobCounter.MinHeight = 10; //*-*-*-*-
blobCounter.MinWidth = 10;
blobCounter.MaxHeight = 50;
blobCounter.MaxWidth = 50;
blobCounter.ProcessImage(bitmapData);
Blob[] blobs = blobCounter.GetObjectsInformation();
grayImage.UnlockBits(bitmapData);
// step 3 - check objects' type and highlight
SimpleShapeChecker shapeChecker = new SimpleShapeChecker();
Graphics g = Graphics.FromImage(pic);
Pen redPen = new Pen(Color.Red, 2); // quadrilateral
Pen bluePen = new Pen(Color.Blue, 2); // triangle
for (int i = 0, n = blobs.Length; i < n; i++)
{
List<IntPoint> edgePoints = blobCounter.GetBlobsEdgePoints(blobs[i]);
List<IntPoint> corners;
// is triangle or quadrilateral
if (shapeChecker.IsQuadrilateral(edgePoints, out corners))
{
// get sub-type
PolygonSubType subType = shapeChecker.CheckPolygonSubType(corners);
Pen pen;
if (subType == PolygonSubType.Square)
{
pen = (corners.Count == 4) ? bluePen : redPen;
g.DrawPolygon(pen, ToPointsArray(corners));
}
}
}
redPen.Dispose();
bluePen.Dispose();
g.Dispose();
pictureBox1.Image = pic;
Problem is low accuracy on detecting squares and rectangles!!!
How can I solve this problem?

If you can use OpenCV this is quite easy. Use the Hough transform and find peaks in the output. Those correspond to straight lines in your input.
If you can't use OpenCV, you'll need to implement it yourself. Here's something to get you started.
https://en.wikipedia.org/?title=Hough_transform
EDIT
As Anders Gustafsson points out in the comment below, the Hough Transform is available in AForge for .NET so implementing it yourself isn't a necessity.
http://www.aforgenet.com/framework/features/hough_transformation.html

Using Seq in emgu c#

I have a self-defined struct of a point (as opposed to System.Drawing.Point):
struct PointD
{
public double X,Y;
}
I want to get a Seq of points, and from there extract the minimum area rectangle:
using (MemStorage stor = new MemStorage())
{
Seq<PointD> seq = new Seq<PointD>(CvInvoke.CV_MAKETYPE(6, 2), stor);
seq.Push(new PointD(0.5, 0));
seq.Push(new PointD(1.0, 0));
seq.Push(new PointD(0, 1.0));
seq.Push(new PointD(1.0, 1.0));
var output = seq.GetMinAreaRect();
}
However, this code throws an exception at GetMinAreaRect(), saying that the input sequence must be 2d points. My question is there a way to get my data format through correctly? I was thinking that I just lack a bit of something, as this code can work for a System.Drawing.Point.

I assume following would work:
int[] pts = {
new PointF(0.5, 0),
new PointF(1.0, 0),
new PointF(0, 1.0),
new PointF(1.0, 1.0)
};
MCvBox2D box = PointCollection.MinAreaRect(pts);
Additional example : http://www.emgu.com/wiki/index.php/Minimum_Area_Rectangle_in_CSharp