Develop Reference

C# Emgucv CvInvoke.DrawContours run very slow

I am using DrawContours method in emgucv package, but it's working very slow, it takes about "12" second
I do the same thing in python, it only takes maybe "2" second to do this.
The target is "Find out all external contour areas that is bigger than 800 pixels and Fill it up on a new canvas" (The number of contours here is 74 )
than I will use this image as a mask
Please tell me how can I improve the time consuming problem
Here's the C# code :
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
CvInvoke.FindContours(img, contours, null, RetrType.External, ChainApproxMethod.ChainApproxSimple);
UMat mask = new UMat(img.Size, DepthType.Cv8U, 1);
int minArea = 800;
for (int i = 0; i < contours.Size; i++)
{
int area = CvInvoke.ContourArea(contours[i]);
if (area > minArea)
{
CvInvoke.DrawContours(mask, contours, i, new MCvScalar(255), -1);
}
else
continue;
}
also show my python code :
contours,_ = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
C = len(contours)
for i in range(0, C):
if cv2.contourArea(contours[i]) > 800 :
cv2.drawContours(mask, contours, i, (255,255,255), -1)
else:
continue
Thank you

CrossCorrelation of images

I used the Accord Framework to implement the cross correlation between two images. My goal is to find by how much (how many pixels, and in which direction) the second image is shifted compared to the first one.
The basic formula I used is the following :
corr(a, b) = ifft(fft(a_and_zeros) * conj(fft(b_and_zeros)))
I'll put the whole code at the end of my message, everything happens on a Click event. My initial images were stored in 1024*768 bitmaps. So here is the steps I have taken :
I cropped the 2 images into the 4 zones that were interesting for me (ExpInit, ExpFinal, RefInit, RefFinal), that I want to correlate two by two (ExpInit with ExpFinal and RefInit with RefFinal). Those cropped images have dimensions of 1024*131.
I put those cropped images in the center of new bitmaps with 2^n dimensions (2048*512).
Applied a Grayscaling filter to get 8bppIndexed PixelFormat.
Converted each image to ComplexImageformat and applied forward FFT on the 4 images.
Complex-conjugating every elements in the RefFinal and ExpFinal fourier-transformed ComplexImage.
Execute element-wise multiplication between the ComplexImageobjects I want to cross-correlate (ExpInit with ExpFinal, RefInit wit RefFinal).
Apply the backward FFT to the product of the element-wise multiplication. Tadaaa, my cross-correlation is done, and I have two Complex[,] objects with the dimensions of my images (2048*512 pixels)
Now I want to answer my initial question : by how much (how many pixels, and in which direction) is the ExpFinal (respectively RefFinal) image shifted compared to the ExpInit (respectively ExpFinal). Here I am left puzzled.
I have the intutition I should be drawing a 3D graph with my Complex[,] object, where x and y are the index in the array, and z the value at the index, and search for the max value, but how do I do that with complex numbers ? Do I use only the Re part ? Only the Im part ? The amplitude ? Or am I completely mistaken ?
Bonus question : what is a good library for drawing such graphs ?
Here is the whole code for the described cross-correlation :
private void crosscorrButton_Click(object sender, EventArgs e)
{
// Cropping all 4 sections (RefInit, ExpInit, RefFinal, ExpFinal) and placing them in the center of new Bitmaps with 2^n dimensions
Rectangle rExp = new Rectangle(1, 157, 1024, 131);
Bitmap ExpInitCrop = new Bitmap(rExp.Width, rExp.Height);
Graphics g = Graphics.FromImage(ExpInitCrop);
g.DrawImage(BMInit, -rExp.X, -rExp.Y);
Bitmap ExpInitLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(ExpInitLarge))
{
largeGraphics.DrawImage(ExpInitCrop, 513, 190);
}
Rectangle rRef = new Rectangle(1, 484, 1024, 131);
Bitmap RefInitCrop = new Bitmap(rRef.Width, rRef.Height);
Graphics h = Graphics.FromImage(RefInitCrop);
h.DrawImage(BMInit, -rRef.X, -rRef.Y);
Bitmap RefInitLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(RefInitLarge))
{
largeGraphics.DrawImage(RefInitCrop, 513, 190);
}
Bitmap ExpFinalCrop = new Bitmap(rExp.Width, rExp.Height);
Graphics i = Graphics.FromImage(ExpFinalCrop);
i.DrawImage(BMFinal, -rExp.X, -rExp.Y);
Bitmap ExpFinalLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(ExpFinalLarge))
{
largeGraphics.DrawImage(ExpFinalCrop, 513, 190);
}
Bitmap RefFinalCrop = new Bitmap(rRef.Width, rRef.Height);
Graphics j = Graphics.FromImage(RefFinalCrop);
j.DrawImage(BMFinal, -rRef.X, -rRef.Y);
Bitmap RefFinalLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(RefFinalLarge))
{
largeGraphics.DrawImage(RefFinalCrop, 513, 190);
}
// Grayscalling the 4 sections to get 8bppIndexed PixelFormat
Accord.Imaging.Filters.Grayscale filterGS = new Accord.Imaging.Filters.Grayscale(0.2125, 0.7154, 0.0721);
Bitmap RefFinalLargeGS = filterGS.Apply(RefFinalLarge);
Bitmap ExpFinalLargeGS = filterGS.Apply(ExpFinalLarge);
Bitmap RefInitLargeGS = filterGS.Apply(RefInitLarge);
Bitmap ExpInitLargeGS = filterGS.Apply(ExpInitLarge);
// FFT on the 4 sections
Accord.Imaging.ComplexImage ExpInitComplex = Accord.Imaging.ComplexImage.FromBitmap(ExpInitLargeGS);
ExpInitComplex.ForwardFourierTransform();
Accord.Imaging.ComplexImage RefInitComplex = Accord.Imaging.ComplexImage.FromBitmap(RefInitLargeGS);
RefInitComplex.ForwardFourierTransform();
Accord.Imaging.ComplexImage ExpFinalComplex = Accord.Imaging.ComplexImage.FromBitmap(ExpFinalLargeGS);
ExpFinalComplex.ForwardFourierTransform();
Accord.Imaging.ComplexImage RefFinalComplex = Accord.Imaging.ComplexImage.FromBitmap(RefFinalLargeGS);
RefFinalComplex.ForwardFourierTransform();
//Conjugating the ExpFinal and RefFinal section
Complex[,] CompConjExpFinal = new Complex[ExpFinalComplex.Height, ExpFinalComplex.Width];
Complex[,] CompConjRefFinal = new Complex[RefFinalComplex.Height, RefFinalComplex.Width];
for (int l = 0; l < ExpFinalComplex.Height; l++)
{
for (int m = 0; m < ExpFinalComplex.Width; m++)
{
CompConjExpFinal[l, m] = System.Numerics.Complex.Conjugate(ExpFinalComplex.Data[l, m]);
ExpFinalComplex.Data[l, m] = CompConjExpFinal[l, m];
}
}
for (int l = 0; l < RefFinalComplex.Height; l++)
{
for (int m = 0; m < RefFinalComplex.Width; m++)
{
CompConjRefFinal[l, m] = System.Numerics.Complex.Conjugate(RefFinalComplex.Data[l, m]);
RefFinalComplex.Data[l, m] = CompConjRefFinal[l, m];
}
}
//Element-wise multiplication of the complex arrays two by two
Complex[,] ExpMultipliedMatrix = new Complex[ExpFinalComplex.Height, ExpFinalComplex.Width];
Complex[,] RefMultipliedMatrix = new Complex[RefFinalComplex.Height, RefFinalComplex.Width];
for (int l = 0; l < ExpFinalComplex.Height; l++)
{
for (int m = 0; m < ExpFinalComplex.Width; m++)
{
ExpMultipliedMatrix[l, m] = System.Numerics.Complex.Multiply(ExpInitComplex.Data[l, m], ExpFinalComplex.Data[l, m]);
RefMultipliedMatrix[l, m] = System.Numerics.Complex.Multiply(RefInitComplex.Data[l, m], RefFinalComplex.Data[l, m]);
}
}
//InverseFFT
Complex[,] CrossCorrExpMatrix = new Complex[ExpFinalComplex.Height, ExpFinalComplex.Width];
Complex[,] CrossCorrRefMatrix = new Complex[RefFinalComplex.Height, RefFinalComplex.Width];
Accord.Math.FourierTransform.FFT2(ExpMultipliedMatrix, FourierTransform.Direction.Backward);
Accord.Math.FourierTransform.FFT2(RefMultipliedMatrix, FourierTransform.Direction.Backward);
CrossCorrExpMatrix = ExpMultipliedMatrix;
CrossCorrRefMatrix = RefMultipliedMatrix;
}
Thanks a lot !

The imaginary part of the result should be 0 (or within numerical error). To find the shift you should be looking at the location of the peak of the correlation's amplitude (but unless you've got one is the negative image of the other, that's likely to correspond to the peak of the correlation's real part). The main thing to be careful about: since you centered both images, an extra shift (of half the image size) will be introduced.
As for viewing the graph, you could fairly easily map the result to a grayscale image and view it with your favorite image viewer.

Find Black Squares/Rectangles in Image with C#

I want to find small black squares/rectangles in scanned sheet to:
deskewing image if needed.
remove white page margin.
Input image:
sample image 1 http://us.cdn.persiangig.com/preview/ii2jf6/2.jpg
Output image:
sample image 2 http://us.cdn.persiangig.com/preview/9ntpnc/1.jpg
My code to find square is:
Bitmap pic =(Bitmap) pictureBox1.Image;
// create filter
AForge.Imaging.Filters.Median Medianfilter = new AForge.Imaging.Filters.Median();
// apply the filter
Medianfilter.ApplyInPlace(pic);
Bitmap grayImage;
if (pic.PixelFormat != PixelFormat.Format16bppGrayScale && pic.PixelFormat != PixelFormat.Format8bppIndexed)
{
AForge.Imaging.Filters.Grayscale grayscalefilter = new AForge.Imaging.Filters.Grayscale(0.2125, 0.7154, 0.0721);
grayImage = grayscalefilter.Apply((Bitmap)pictureBox1.Image);
}
else
{
grayImage = pic;
}
// black & white:
Threshold(ref grayImage, Convert.ToInt32(textBox1.Text), Convert.ToInt32(textBox1.Text));
// invert filter
Invert invertfilter = new Invert();
// apply the filter
invertfilter.ApplyInPlace(grayImage);
// Edge Detector filter
DifferenceEdgeDetector EdgeDetectorfilter = new DifferenceEdgeDetector();
// apply the filter
EdgeDetectorfilter.ApplyInPlace(grayImage);
// create filter
Dilatation Dilatationfilter = new Dilatation();
// apply the filter
Dilatationfilter.ApplyInPlace(grayImage);
Finding object(square/rectangle) in image:
// lock image
BitmapData bitmapData = grayImage.LockBits(new Rectangle(0, 0, grayImage.Width, grayImage.Height),
ImageLockMode.ReadWrite, grayImage.PixelFormat);
// step 2 - locating objects
BlobCounter blobCounter = new BlobCounter();
blobCounter.FilterBlobs = true;
blobCounter.MinHeight = 10; //*-*-*-*-
blobCounter.MinWidth = 10;
blobCounter.MaxHeight = 50;
blobCounter.MaxWidth = 50;
blobCounter.ProcessImage(bitmapData);
Blob[] blobs = blobCounter.GetObjectsInformation();
grayImage.UnlockBits(bitmapData);
// step 3 - check objects' type and highlight
SimpleShapeChecker shapeChecker = new SimpleShapeChecker();
Graphics g = Graphics.FromImage(pic);
Pen redPen = new Pen(Color.Red, 2); // quadrilateral
Pen bluePen = new Pen(Color.Blue, 2); // triangle
for (int i = 0, n = blobs.Length; i < n; i++)
{
List<IntPoint> edgePoints = blobCounter.GetBlobsEdgePoints(blobs[i]);
List<IntPoint> corners;
// is triangle or quadrilateral
if (shapeChecker.IsQuadrilateral(edgePoints, out corners))
{
// get sub-type
PolygonSubType subType = shapeChecker.CheckPolygonSubType(corners);
Pen pen;
if (subType == PolygonSubType.Square)
{
pen = (corners.Count == 4) ? bluePen : redPen;
g.DrawPolygon(pen, ToPointsArray(corners));
}
}
}
redPen.Dispose();
bluePen.Dispose();
g.Dispose();
pictureBox1.Image = pic;
Problem is low accuracy on detecting squares and rectangles!!!
How can I solve this problem?

If you can use OpenCV this is quite easy. Use the Hough transform and find peaks in the output. Those correspond to straight lines in your input.
If you can't use OpenCV, you'll need to implement it yourself. Here's something to get you started.
https://en.wikipedia.org/?title=Hough_transform
EDIT
As Anders Gustafsson points out in the comment below, the Hough Transform is available in AForge for .NET so implementing it yourself isn't a necessity.
http://www.aforgenet.com/framework/features/hough_transformation.html

Matrix / coordinate transformation in C#

I have an array of coordinates that reflect known positions on an image. Let's call this the template image. It has a unique barcode and orientation markers (which are also in the coordinate array).
The image is printed, scanned and fed back into my application to be detected. During printing and scanning, the image could be transformed in three ways; translation, rotation and scale.
Assuming that I can find the orientation markers on the distorted image, how can I use matrix transformation to get the relative positions of the remaining coordinates?
I posted this question on SO before but made it too complicated to understand what I wanted.
EDIT
namespace MatrixTest
{
using System;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Collections.Generic;
public static class Program
{
public static void Main ()
{
Template template = new Template(); // Original template image.
Document document = new Document(); // Printed and scanned distorted image.
template.CreateTemplateImage();
// The template image is printed and scanned. This method generates an example scan or this question.
document.CreateDistortedImageFromTemplateImage();
// Stuck here.
document.Transform();
// Draw transformed points on the image to verify that transformation is successful.
document.DrawPoints();
System.Diagnostics.Process.Start(new System.IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly().Location).Directory.FullName);
}
}
public class Page
{
public Bitmap Image { get; set; }
public Point[] Markers = new Point[3]; // Orientation markers: 1=TopLeft, 2=TopRight, 3=BottomRight.
public Point[] Points = new Point[100]; // Coordinates to transform in the TemplateScanned derived class!
}
// This class represents the originalk template image.
public class Template: Page
{
public Template ()
{
this.Image = new Bitmap(300, 400);
// Known dimentions for marker rectangles.
this.Markers[0] = new Point(10, 10);
this.Markers[1] = new Point(this.Image.Width - 20 - 10, 10);
this.Markers[2] = new Point(this.Image.Width - 20 - 10, this.Image.Height - 20 - 10);
// Known points of interest. Consider them hardcoded.
int index = 0;
for (int y = 0; y < 10; y++)
for (int x = 0; x < 10; x++)
this.Points[index++] = new Point((this.Image.Width / 10) + (x * 20), (this.Image.Height / 10) + (y * 20));
}
public void CreateTemplateImage ()
{
using (Graphics graphics = Graphics.FromImage(this.Image))
{
graphics.Clear(Color.White);
for (int i = 0; i < this.Markers.Length; i++)
graphics.FillRectangle(Brushes.Black, this.Markers[i].X, this.Markers[i].Y, 20, 20);
for (int i = 0; i < this.Points.Length; i++)
graphics.DrawRectangle(Pens.Red, this.Points[i].X, this.Points[i].Y, 5, 5);
}
this.Image.Save("Document Original.png");
}
}
// This class represents the scanned image.
public class Document: Page
{
public struct StructTransformation
{
public float AngleOfRotation;
public SizeF ScaleRatio;
public SizeF TranslationOffset;
}
private Template Template = new Template();
private StructTransformation Transformation = new StructTransformation();
public Document ()
{
this.Template = new Template();
this.Transformation = new StructTransformation { AngleOfRotation = 5f, ScaleRatio = new SizeF(.8f, .7f), TranslationOffset = new SizeF(100f, 30f) };
this.Template.CreateTemplateImage();
// Copy points from template.
for (int i = 0; i < this.Template.Markers.Length; i++)
this.Markers[i] = this.Template.Markers[i];
for (int i = 0; i < this.Points.Length; i++)
this.Points[i] = this.Template.Points[i];
}
// Just distorts the original template image as if it had been read from a scanner.
public void CreateDistortedImageFromTemplateImage ()
{
// Distort coordinates.
Matrix matrix = new Matrix();
matrix.Rotate(this.Transformation.AngleOfRotation);
matrix.Scale(this.Transformation.ScaleRatio.Width, this.Transformation.ScaleRatio.Height);
matrix.Translate(this.Transformation.TranslationOffset.Width, this.Transformation.TranslationOffset.Height);
matrix.TransformPoints(this.Markers);
matrix.TransformPoints(this.Points);
// Distort and save image for visual reference.
this.Image = new Bitmap(this.Template.Image.Width, this.Template.Image.Height);
using (Graphics graphics = Graphics.FromImage(this.Image))
{
graphics.Clear(Color.White);
graphics.RotateTransform(this.Transformation.AngleOfRotation);
graphics.ScaleTransform(this.Transformation.ScaleRatio.Width, this.Transformation.ScaleRatio.Height);
graphics.TranslateTransform(this.Transformation.TranslationOffset.Width, this.Transformation.TranslationOffset.Height);
graphics.DrawImage(this.Template.Image, 0, 0);
}
this.Image.Save("Document Scanned.png");
}
public void Transform ()
{
// The rectangles of the ScannedDcoument are not known at this time. They would obviously be relative to the three orientation markers.
// I can't figure out how to use the following code properly i.e. using Matrix to apply all three transformations.
Matrix matrix = new Matrix();
matrix.Rotate(-this.Transformation.AngleOfRotation);
matrix.Scale(1f/this.Transformation.ScaleRatio.Width, 1f/this.Transformation.ScaleRatio.Height);
matrix.Translate(-this.Transformation.TranslationOffset.Width, -this.Transformation.TranslationOffset.Height);
matrix.TransformPoints(this.Markers);
matrix.TransformPoints(this.Points);
}
public void DrawPoints ()
{
using (Graphics graphics = Graphics.FromImage(this.Image))
{
graphics.Clear(Color.White);
for (int i = 0; i < this.Markers.Length; i++)
graphics.FillRectangle(Brushes.Blue, this.Markers[i].X, this.Markers[i].Y, 20, 20);
for (int i = 0; i < this.Points.Length; i++)
graphics.DrawRectangle(Pens.Purple, this.Points[i].X, this.Points[i].Y, 5, 5);
}
this.Image.Save("Document Fixed.png");
}
}
}

I'm assuming you want to transform the image to the unit square ( (0, 0) - (1.0, 1.0))
You need three points, one is the origin, the other will be transformed to the x-axis (1.0, 0) and the other to the y-axis (0, 1.0).
In the original coordinate system:
The origin is (Ox, Oy)
The X-axis is (X1, Y2)
The Y-Axis is (X2, Y2)
X-Axis relative to the origin (X1-Ox, Y1-Oy) will be shortened to (RX1, RY1)
Y-Axis relative to the origin (X2-ox, Y2-Oy) will be shortened to (RX2, RY2)
First we will shift the origin to (0,0) in homogeneous coordinates the transform matrix will be
(1 0 -Ox)
(0 1 -Oy)
(0 0 1)
The transform from the new space to the old one is represented by the following matrix:
(RX1 RX2 0)
(RY1 RY2 0)
( 0 0 1)
Because you want the inverse transformation, from old space to the new one, we need the invert this matrix:
Let's shorten (RX1*RY2-RX2*RY1) as D
(RY2/D -RX2/D 0)
(-RY1/D RX1/D 0)
( 0 0 1)
Now you can multiply both matrices first you do the translation and then use the second matrix to transform the basis.

Fill the holes in emgu cv

How can I fill the holes in binary image in emgu cv?
In Aforge.net it's easy, use Fillholes class.

Thought the question is a little bit old, I'd like to contribute an alternative solution to the problem.
You can obtain the same result as Chris' without memory problem if you use the following:
private Image<Gray,byte> FillHoles(Image<Gray,byte> image)
{
var resultImage = image.CopyBlank();
Gray gray = new Gray(255);
using (var mem = new MemStorage())
{
for (var contour = image.FindContours(
CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
RETR_TYPE.CV_RETR_CCOMP,
mem); contour!= null; contour = contour.HNext)
{
resultImage.Draw(contour, gray, -1);
}
}
return resultImage;
}
The good thing about the method above is that you can selectively fill holes that meets your criteria. For example, you may want to fill holes whose pixel count (count of black pixels inside the blob) is below 50, etc.
private Image<Gray,byte> FillHoles(Image<Gray,byte> image, int minArea, int maxArea)
{
var resultImage = image.CopyBlank();
Gray gray = new Gray(255);
using (var mem = new MemStorage())
{
for (var contour = image.FindContours(
CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
RETR_TYPE.CV_RETR_CCOMP,
mem); contour!= null; contour = contour.HNext)
{
if ( (contour.Area < maxArea) && (contour.Area > minArea) )
resultImage.Draw(contour, gray, -1);
}
}
return resultImage;
}

Yes there is a method but it's a bit messy as its based on cvFloodFill operation. Now all this algorithm is designed to do is fill an area with a colour until it reaches an edge similar to a region growing algorithm. To use this effectively you need to use a little inventive coding but I warn you this code is only to get you started it may require re-factoring to speed things up . As it stands the loop goes through each of your pixels that are less then 255 applies cvFloodFill checks what size the area is and then if it is under a certain area fill it in.
It is important to note that a copy of the image is made of the original image to be supplied to the cvFloodFill operation as a pointer is used. If the direct image is supplied then you will end up with a white image.
OpenFileDialog OpenFile = new OpenFileDialog();
if (OpenFileDialog.ShowDialog() == DialogResult.OK)
{
Image<Bgr, byte> image = new Image<Bgr, byte>(OpenFile.FileName);
for (int i = 0; i < image.Width; i++)
{
for (int j = 0; j < image.Height; j++)
{
if (image.Data[j, i, 0] != 255)
{
Image<Bgr, byte> image_copy = image.Copy();
Image<Gray, byte> mask = new Image<Gray, byte>(image.Width + 2, image.Height + 2);
MCvConnectedComp comp = new MCvConnectedComp();
Point point1 = new Point(i, j);
//CvInvoke.cvFloodFill(
CvInvoke.cvFloodFill(image_copy.Ptr, point1, new MCvScalar(255, 255, 255, 255),
new MCvScalar(0, 0, 0),
new MCvScalar(0, 0, 0), out comp,
Emgu.CV.CvEnum.CONNECTIVITY.EIGHT_CONNECTED,
Emgu.CV.CvEnum.FLOODFILL_FLAG.DEFAULT, mask.Ptr);
if (comp.area < 10000)
{
image = image_copy.Copy();
}
}
}
}
}
The "new MCvScalar(0, 0, 0), new MCvScalar(0, 0, 0)," are not really important in this case as you are only filling in results of a binary image. YOu could play around with other settings to see what results you can achieve. "if (comp.area < 10000)" is the key constant to change is you want to change what size hole the method will fill.
These are the results that you can expect:
Original
Results
The problem with this method is it's extremely memory intensive and it managed to eat up 6GB of ram on a 200x200 image and when I tried 200x300 it ate all 8GB of my RAM and brought everything to a crashing halt. Unless a majority of your image is white and you want to fill in tiny gaps or you can minimise where you apply the method I would avoid it. I would suggest writing you own class to examine each pixel that is not 255 and add the number of pixels surrounding it. You can then record the position of each pixel that was not 255 (in a simple list) and if your count was bellow a threshold set these positions to 255 in your images (by iterating though the list).
I would stick with the Aforge FillHoles class if you do not wish to write your own as it is designed for this purpose.
Cheers
Chris

you can use FillConvexPoly
image.FillConvexPoly(externalContours.ToArray(), new Gray(255));