Develop Reference

OpenCV FindContours does not find the correct rectangle

Using CvInvoke.Canny and CvInvoke.FindContours I'm trying to find the rectangle containing the item name (Schematic: Maple Desk). This rectangle is shown in the image below:
I'm able to find a lot of rectangles but I'm not able to get this one. Tried a lot of different thresholds for Canny but to no effect. The following image shows all rectangles I currently get:
Any ideas how to tackle this? Do I need to use other thresholds or another approach? I already experimented using grayscale and blurring but that didn't give better result. I added my current source below and the original image I'm using is this:
public Mat ProcessImage(Mat img)
{
UMat filter = new UMat();
UMat cannyEdges = new UMat();
Mat rectangleImage = new Mat(img.Size, DepthType.Cv8U, 3);
//Convert the image to grayscale and filter out the noise
//CvInvoke.CvtColor(img, filter, ColorConversion.Bgr2Gray);
//Remove noise
//CvInvoke.GaussianBlur(filter, filter, new System.Drawing.Size(3, 3), 1);
// Canny and edge detection
double cannyThreshold = 1.0; //180.0
double cannyThresholdLinking = 1.0; //120.0
//CvInvoke.Canny(filter, cannyEdges, cannyThreshold, cannyThresholdLinking);
CvInvoke.Canny(img, cannyEdges, cannyThreshold, cannyThresholdLinking);
// Find rectangles
List<RotatedRect> rectangleList = new List<RotatedRect>();
using (VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint())
{
CvInvoke.FindContours(cannyEdges, contours, null, RetrType.List, ChainApproxMethod.ChainApproxSimple);
int count = contours.Size;
for (int i = 0; i < count; i++)
{
using (VectorOfPoint contour = contours[i])
using (VectorOfPoint approxContour = new VectorOfPoint())
{
CvInvoke.ApproxPolyDP(contour, approxContour, CvInvoke.ArcLength(contour, true) * 0.05, true);
// Only consider contours with area greater than 250
if (CvInvoke.ContourArea(approxContour, false) > 250)
{
// The contour has 4 vertices.
if (approxContour.Size == 4)
{
// Determine if all the angles in the contour are within [80, 100] degree
bool isRectangle = true;
System.Drawing.Point[] pts = approxContour.ToArray();
LineSegment2D[] edges = Emgu.CV.PointCollection.PolyLine(pts, true);
for (int j = 0; j < edges.Length; j++)
{
double angle = Math.Abs(edges[(j + 1) % edges.Length].GetExteriorAngleDegree(edges[j]));
if (angle < 80 || angle > 100)
{
isRectangle = false;
break;
}
}
if (isRectangle) rectangleList.Add(CvInvoke.MinAreaRect(approxContour));
}
}
}
}
}
// Draw rectangles
foreach (RotatedRect rectangle in rectangleList)
{
CvInvoke.Polylines(rectangleImage, Array.ConvertAll(rectangle.GetVertices(), System.Drawing.Point.Round), true,
new Bgr(Color.DarkOrange).MCvScalar, 2);
}
//Drawing a light gray frame around the image
CvInvoke.Rectangle(rectangleImage,
new Rectangle(System.Drawing.Point.Empty,
new System.Drawing.Size(rectangleImage.Width - 1, rectangleImage.Height - 1)),
new MCvScalar(120, 120, 120));
//Draw the labels
CvInvoke.PutText(rectangleImage, "Rectangles", new System.Drawing.Point(20, 20),
FontFace.HersheyDuplex, 0.5, new MCvScalar(120, 120, 120));
Mat result = new Mat();
CvInvoke.VConcat(new Mat[] { img, rectangleImage }, result);
return result;
}
Edit 1:
After some more fine tuning with the following thresholds for Canny
cannyThreshold 100
cannyThresholdLinking 400
And using a minimum size for all ContourAreas of 10000 I can get the following result:
Edit 2:
For those interested, solved using the current detection, no changes were needed. Used the 2 detected rectangles in the screenshot above to get the location of the missing rectangle containing the item name.
Result can be found here:
https://github.com/josdemmers/NewWorldCompanion

For those interested, solved using the current detection, no changes were needed. Used the 2 detected rectangles in the screenshot above to get the location of the missing rectangle containing the item name.
Result can be found here: https://github.com/josdemmers/NewWorldCompanion

Why is AForge.net giving a different output in case of FFT Auto-correlation?

See this related question.
I want to obtain the same outcome using AForge.net framework. The output should match the following:
The output seems to be not coming as expected:
Why is the output different in AForge.net?
.
Source Code
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
Bitmap image = (Bitmap)Bitmap.FromFile(#"StandardImage\\lena.png");
Bitmap conv = new Bitmap(image.Width, image.Height, image.PixelFormat);
ComplexImage cImage = ComplexImage.FromBitmap(image);
cImage.ForwardFourierTransform();
ComplexImage cKernel = ComplexImage.FromBitmap(image);
cImage.ForwardFourierTransform();
ComplexImage convOut = ComplexImage.FromBitmap(conv);
convOut.ForwardFourierTransform();
for (int y = 0; y < cImage.Height; y++)
{
for (int x = 0; x < cImage.Width; x++)
{
convOut.Data[x, y] = cImage.Data[x, y] * cKernel.Data[x, y];
}
}
convOut.BackwardFourierTransform();
Bitmap bbbb = convOut.ToBitmap();
pictureBox1.Image = bbbb;
}
}

The main problem is
ComplexImage cKernel = ComplexImage.FromBitmap(image);
//cImage.ForwardFourierTransform(); //<--- This line should be FFT of cKernel
cKernel.ForwardFourierTransform();
This would solve the problem you mentioned in the resulting image, but if you want to get an image similar to the bottom right image you need to do some normalization to increase the intensity of pixels.
update:
The bottom right image is actually a Fourier image so I think we should remove the BFF.
//convOut.BackwardFourierTransform();

It is seems like you are not using a gaussian kernel with Aforge,
Anyway, the library has a method for convolution with gaussian:
int w=4,h=11;
GaussianBlur filter = new GaussianBlur( w, h );
// apply the filter
filter.ApplyInPlace( image );
Try it and the output should be the same as the others.

Plotting 2D heat map

I have a chart on which I want to plot a heat map; the only data I have is humidity and temperature, which represent a point in the chart.
How do I get the rectangular type of heat map on the chart in c#?
What I want is similar to picture below :
What I really want is a rectangular region in the chart which is plotted in different color based on the point that i get from the list of points and form the colorful section in the chart.

You have a choice of at least three ways to create a chart with colored rectangles that make up a heat map.
Here is one example
that uses/abuses a DataGridView. While I would not suggest this, the post contains a useful function that creates nice color lists to use in your task.
Then there is the option to draw the chart using GDI+ methods, namely Graphics.FillRectangle. This not hard at all but once you want to get those nice extras a Chart control offers, like scaling, axes, tooltips etc the work adds up.. See below!
So let's have a look at option three: Using the Chart control from the DataVisualization namespace.
Let's first assume that you have created a list of colors:
List<Color> colorList = new List<Color>();
And that you have managed to project your data onto a 2D array of int indices that point into the color list:
int[,] coloredData = null;
Next you have to pick a ChartType for your Series S1 There really is only one I can think of that will help:
S1.ChartType = SeriesChartType.Point;
Points are displayed by Markers. We want the DataPoints not really displayed as one of the standard MarkerTypes.
Square would be ok, if we wanted to display squares; but for rectangles it will not work well: Even if we let them overlap there will still be points at the borders that have a different size because they don't fully overlap..
So we use a custom marker by setting the MarkerImage of each point to a bitmap of a suitable size and color.
Here is a loop that adds the DataPoints to our Series and sets each to have a MarkerImage:
for (int x = 1; x < coloredData.GetLength(0); x++)
for (int y = 1; y < coloredData.GetLength(1); y++)
{
int pt = S1.Points.AddXY(x, y);
S1.Points[pt].MarkerImage = "NI" + coloredData[x,y];
}
This takes some explaining: To set a MarkerImage that is not at a path on the disk, it has to reside in the Chart's Images collection. This means is needs to be of type NamedImage. Any image will do, but it has to have a unique name string added to identify it in the NamedImagesCollection . I chose the names to be 'NI1', 'NI2'..
Obviously we need to create all those images; here is a function to do that:
void createMarkers(Chart chart, int count)
{
// rough calculation:
int sw = chart.ClientSize.Width / coloredData.GetLength(0);
int sh = chart.ClientSize.Height / coloredData.GetLength(1);
// clean up previous images:
foreach(NamedImage ni in chart1.Images) ni.Dispose();
chart.Images.Clear();
// now create count images:
for (int i = 0; i < count; i++)
{
Bitmap bmp = new Bitmap(sw, sh);
using (Graphics G = Graphics.FromImage(bmp))
G.Clear(colorList[i]);
chart.Images.Add(new NamedImage("NI" + i, bmp));
}
}
We want all markers to have at least roughly the right size; so whenever that size changes we set it again:
void setMarkerSize(Chart chart)
{
int sx = chart1.ClientSize.Width / coloredData.GetLength(0);
int sy = chart1.ClientSize.Height / coloredData.GetLength(1);
chart1.Series["S1"].MarkerSize = (int)Math.Max(sx, sy);
}
This doesn't care much about details like the InnerPlotPosition, i.e. the actual area to draw to; so here is some room for refinement..!
We call this when we set up the chart but also upon resizing:
private void chart1_Resize(object sender, EventArgs e)
{
setMarkerSize(chart1);
createMarkers(chart1, 100);
}
Let's have a look at the result using some cheap testdata:
As you can see resizing works ok..
Here is the full code that set up my example:
private void button6_Click(object sender, EventArgs e)
{
List<Color> stopColors = new List<Color>()
{ Color.Blue, Color.Cyan, Color.YellowGreen, Color.Orange, Color.Red };
colorList = interpolateColors(stopColors, 100);
coloredData = getCData(32, 24);
// basic setup..
chart1.ChartAreas.Clear();
ChartArea CA = chart1.ChartAreas.Add("CA");
chart1.Series.Clear();
Series S1 = chart1.Series.Add("S1");
chart1.Legends.Clear();
// we choose a charttype that lets us add points freely:
S1.ChartType = SeriesChartType.Point;
Size sz = chart1.ClientSize;
// we need to make the markers large enough to fill the area completely:
setMarkerSize(chart1);
createMarkers(chart1, 100);
// now we fill in the datapoints
for (int x = 1; x < coloredData.GetLength(0); x++)
for (int y = 1; y < coloredData.GetLength(1); y++)
{
int pt = S1.Points.AddXY(x, y);
// S1.Points[pt].Color = coloredData[x, y];
S1.Points[pt].MarkerImage = "NI" + coloredData[x,y];
}
}
A few notes on limitations:
The point will always sit on top of any gridlines. If you really needs those you will have to draw them on top in one of the the Paint events.
The labels as shown are referring to the integers indices of the data array. If you want to show the original data, one way would be to add CustomLabels to the axes.. See here for an example!
This should give you an idea of what you can do with a Chart control; to complete your confusion here is how to draw those rectangles in GDI+ using the same colors and data:
Bitmap getChartImg(float[,] data, Size sz, Padding pad)
{
Bitmap bmp = new Bitmap(sz.Width , sz.Height);
using (Graphics G = Graphics.FromImage(bmp))
{
float w = 1f * (sz.Width - pad.Left - pad.Right) / coloredData.GetLength(0);
float h = 1f * (sz.Height - pad.Top - pad.Bottom) / coloredData.GetLength(1);
for (int x = 0; x < coloredData.GetLength(0); x++)
for (int y = 0; y < coloredData.GetLength(1); y++)
{
using (SolidBrush brush = new SolidBrush(colorList[coloredData[x,y]]))
G.FillRectangle(brush, pad.Left + x * w, y * h - pad.Bottom, w, h);
}
}
return bmp;
}
The resulting Bitmap looks familiar:
That was simple; but to add all the extras into the space reserved by the padding will not be so easy..

Is it possible to find the edge of a "spotty" region in emgucv?

I have an image that looks like this:
and I want to find the edges of the dark part so like this (the red lines are what I am looking for):
I have tried a few approaches and none have worked so I am hoping there is an emgu guru out there willing to help me...
Approach 1
Convert the image to grayscale
Remove noise and invert
Remove anything that is not really bright
Get the canny and the polygons
Code for this (I know that I should be disposing of things properly but I am keeping the code short):
var orig = new Image<Bgr, byte>(inFile);
var contours = orig
.Convert<Gray, byte>()
.PyrDown()
.PyrUp()
.Not()
.InRange(new Gray(190), new Gray(255))
.Canny(new Gray(190), new Gray(255))
.FindContours(CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
RETR_TYPE.CV_RETR_TREE);
var output = new Image<Gray, byte>(orig.Size);
for (; contours != null; contours = contours.HNext)
{
var poly = contours.ApproxPoly(contours.Perimeter*0.05,
contours.Storage);
output.Draw(poly, new Gray(255), 1);
}
output.Save(outFile);
This is the result:
Approach 2
Convert the image to grayscale
Remove noise and invert
Remove anything that is not really bright
Get the canny and then lines
Code for this:
var orig = new Image<Bgr, byte>(inFile);
var linesegs = orig
.Convert<Gray, byte>()
.PyrDown()
.PyrUp()
.Not()
.InRange(new Gray(190), new Gray(255))
.Canny(new Gray(190), new Gray(255))
.HoughLinesBinary(
1,
Math.PI/45.0,
20,
30,
10
)[0];
var output = new Image<Gray, byte>(orig.Size);
foreach (var l in linesegs)
{
output.Draw(l, new Gray(255), 1);
}
output.Save(outFile);
This is the result:
Notes
I have tried adjusting all the parameters on those two approaches and adding smoothing but I can never get the simple edges that I need because, I suppose, the darker region is not a solid colour.
I have also tried dilating and eroding but the parameters I have to put in for those are so high to get a single colour that I end up including some of the grey stuff on the right and lose accuracy.

Yes, it's possible, and here is how you could do it:
Change the contrast of the image to make the lighter part disappear:
Then, convert it to HSV to perform a threshold operation on the Saturation channel:
And execute erode & dilate operations to get rid of the noises:
At this point you'll have the result you were looking for. For testing purposes, at the end I execute the bounding box technique to show how to detect the beggining and the end of the area of interest:
I didn't have the time to tweak the parameters and make a perfect detection, but I'm sure you can figure it out. This answer provides a roadmap for achieving that!
This is the C++ code I came up with, I trust you are capable of converting it to C#:
#include <cv.h>
#include <highgui.h>
int main(int argc, char* argv[])
{
cv::Mat image = cv::imread(argv[1]);
cv::Mat new_image = cv::Mat::zeros(image.size(), image.type());
/* Change contrast: new_image(i,j) = alpha*image(i,j) + beta */
double alpha = 1.8; // [1.0-3.0]
int beta = 100; // [0-100]
for (int y = 0; y < image.rows; y++)
{
for (int x = 0; x < image.cols; x++)
{
for (int c = 0; c < 3; c++)
{
new_image.at<cv::Vec3b>(y,x)[c] =
cv::saturate_cast<uchar>(alpha * (image.at<cv::Vec3b>(y,x)[c]) + beta);
}
}
}
cv::imshow("contrast", new_image);
/* Convert RGB Mat into HSV color space */
cv::Mat hsv;
cv::cvtColor(new_image, hsv, CV_BGR2HSV);
std::vector<cv::Mat> v;
cv::split(hsv,v);
// Perform threshold on the S channel of hSv
int thres = 15;
cv::threshold(v[1], v[1], thres, 255, cv::THRESH_BINARY_INV);
cv::imshow("saturation", v[1]);
/* Erode & Dilate */
int erosion_size = 6;
cv::Mat element = cv::getStructuringElement(cv::MORPH_CROSS,
cv::Size(2 * erosion_size + 1, 2 * erosion_size + 1),
cv::Point(erosion_size, erosion_size) );
cv::erode(v[1], v[1], element);
cv::dilate(v[1], v[1], element);
cv::imshow("binary", v[1]);
/* Bounding box */
// Invert colors
cv::bitwise_not(v[1], v[1]);
// Store the set of points in the image before assembling the bounding box
std::vector<cv::Point> points;
cv::Mat_<uchar>::iterator it = v[1].begin<uchar>();
cv::Mat_<uchar>::iterator end = v[1].end<uchar>();
for (; it != end; ++it)
{
if (*it) points.push_back(it.pos());
}
// Compute minimal bounding box
cv::RotatedRect box = cv::minAreaRect(cv::Mat(points));
// Draw bounding box in the original image (debug purposes)
cv::Point2f vertices[4];
box.points(vertices);
for (int i = 0; i < 4; ++i)
{
cv::line(image, vertices[i], vertices[(i + 1) % 4], cv::Scalar(0, 255, 0), 2, CV_AA);
}
cv::imshow("box", image);
cvWaitKey(0);
return 0;
}

Fill the holes in emgu cv

How can I fill the holes in binary image in emgu cv?
In Aforge.net it's easy, use Fillholes class.

Thought the question is a little bit old, I'd like to contribute an alternative solution to the problem.
You can obtain the same result as Chris' without memory problem if you use the following:
private Image<Gray,byte> FillHoles(Image<Gray,byte> image)
{
var resultImage = image.CopyBlank();
Gray gray = new Gray(255);
using (var mem = new MemStorage())
{
for (var contour = image.FindContours(
CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
RETR_TYPE.CV_RETR_CCOMP,
mem); contour!= null; contour = contour.HNext)
{
resultImage.Draw(contour, gray, -1);
}
}
return resultImage;
}
The good thing about the method above is that you can selectively fill holes that meets your criteria. For example, you may want to fill holes whose pixel count (count of black pixels inside the blob) is below 50, etc.
private Image<Gray,byte> FillHoles(Image<Gray,byte> image, int minArea, int maxArea)
{
var resultImage = image.CopyBlank();
Gray gray = new Gray(255);
using (var mem = new MemStorage())
{
for (var contour = image.FindContours(
CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
RETR_TYPE.CV_RETR_CCOMP,
mem); contour!= null; contour = contour.HNext)
{
if ( (contour.Area < maxArea) && (contour.Area > minArea) )
resultImage.Draw(contour, gray, -1);
}
}
return resultImage;
}

Yes there is a method but it's a bit messy as its based on cvFloodFill operation. Now all this algorithm is designed to do is fill an area with a colour until it reaches an edge similar to a region growing algorithm. To use this effectively you need to use a little inventive coding but I warn you this code is only to get you started it may require re-factoring to speed things up . As it stands the loop goes through each of your pixels that are less then 255 applies cvFloodFill checks what size the area is and then if it is under a certain area fill it in.
It is important to note that a copy of the image is made of the original image to be supplied to the cvFloodFill operation as a pointer is used. If the direct image is supplied then you will end up with a white image.
OpenFileDialog OpenFile = new OpenFileDialog();
if (OpenFileDialog.ShowDialog() == DialogResult.OK)
{
Image<Bgr, byte> image = new Image<Bgr, byte>(OpenFile.FileName);
for (int i = 0; i < image.Width; i++)
{
for (int j = 0; j < image.Height; j++)
{
if (image.Data[j, i, 0] != 255)
{
Image<Bgr, byte> image_copy = image.Copy();
Image<Gray, byte> mask = new Image<Gray, byte>(image.Width + 2, image.Height + 2);
MCvConnectedComp comp = new MCvConnectedComp();
Point point1 = new Point(i, j);
//CvInvoke.cvFloodFill(
CvInvoke.cvFloodFill(image_copy.Ptr, point1, new MCvScalar(255, 255, 255, 255),
new MCvScalar(0, 0, 0),
new MCvScalar(0, 0, 0), out comp,
Emgu.CV.CvEnum.CONNECTIVITY.EIGHT_CONNECTED,
Emgu.CV.CvEnum.FLOODFILL_FLAG.DEFAULT, mask.Ptr);
if (comp.area < 10000)
{
image = image_copy.Copy();
}
}
}
}
}
The "new MCvScalar(0, 0, 0), new MCvScalar(0, 0, 0)," are not really important in this case as you are only filling in results of a binary image. YOu could play around with other settings to see what results you can achieve. "if (comp.area < 10000)" is the key constant to change is you want to change what size hole the method will fill.
These are the results that you can expect:
Original
Results
The problem with this method is it's extremely memory intensive and it managed to eat up 6GB of ram on a 200x200 image and when I tried 200x300 it ate all 8GB of my RAM and brought everything to a crashing halt. Unless a majority of your image is white and you want to fill in tiny gaps or you can minimise where you apply the method I would avoid it. I would suggest writing you own class to examine each pixel that is not 255 and add the number of pixels surrounding it. You can then record the position of each pixel that was not 255 (in a simple list) and if your count was bellow a threshold set these positions to 255 in your images (by iterating though the list).
I would stick with the Aforge FillHoles class if you do not wish to write your own as it is designed for this purpose.
Cheers
Chris

you can use FillConvexPoly
image.FillConvexPoly(externalContours.ToArray(), new Gray(255));