How can I compute the DFT of an image (using EMGU), display it and then compute the reverse to get back to the original?
I'm going to answer my own question here since it took me a while to figure out.
To test that it works here's an image
and here's the expected result after applying DFT.
And without further ado here's the code:
// Load image
Image<Gray, float> image = new Image<Gray, float>(#"C:\Users\me\Desktop\lines.png");
// Transform 1 channel grayscale image into 2 channel image
IntPtr complexImage = CvInvoke.cvCreateImage(image.Size, Emgu.CV.CvEnum.IPL_DEPTH.IPL_DEPTH_32F, 2);
CvInvoke.cvSetImageCOI(complexImage, 1); // Select the channel to copy into
CvInvoke.cvCopy(image, complexImage, IntPtr.Zero);
CvInvoke.cvSetImageCOI(complexImage, 0); // Select all channels
// This will hold the DFT data
Matrix<float> forwardDft = new Matrix<float>(image.Rows, image.Cols, 2);
CvInvoke.cvDFT(complexImage, forwardDft, Emgu.CV.CvEnum.CV_DXT.CV_DXT_FORWARD, 0);
CvInvoke.cvReleaseImage(ref complexImage);
// We'll display the magnitude
Matrix<float> forwardDftMagnitude = GetDftMagnitude(forwardDft);
SwitchQuadrants(ref forwardDftMagnitude);
// Now compute the inverse to see if we can get back the original
Matrix<float> reverseDft = new Matrix<float>(forwardDft.Rows, forwardDft.Cols, 2);
CvInvoke.cvDFT(forwardDft, reverseDft, Emgu.CV.CvEnum.CV_DXT.CV_DXT_INV_SCALE, 0);
Matrix<float> reverseDftMagnitude = GetDftMagnitude(reverseDft);
pictureBox1.Image = image.ToBitmap();
pictureBox2.Image = Matrix2Bitmap(forwardDftMagnitude);
pictureBox3.Image = Matrix2Bitmap(reverseDftMagnitude);
private Bitmap Matrix2Bitmap(Matrix<float> matrix)
{
CvInvoke.cvNormalize(matrix, matrix, 0.0, 255.0, Emgu.CV.CvEnum.NORM_TYPE.CV_MINMAX, IntPtr.Zero);
Image<Gray, float> image = new Image<Gray, float>(matrix.Size);
matrix.CopyTo(image);
return image.ToBitmap();
}
// Real part is magnitude, imaginary is phase.
// Here we compute log(sqrt(Re^2 + Im^2) + 1) to get the magnitude and
// rescale it so everything is visible
private Matrix<float> GetDftMagnitude(Matrix<float> fftData)
{
//The Real part of the Fourier Transform
Matrix<float> outReal = new Matrix<float>(fftData.Size);
//The imaginary part of the Fourier Transform
Matrix<float> outIm = new Matrix<float>(fftData.Size);
CvInvoke.cvSplit(fftData, outReal, outIm, IntPtr.Zero, IntPtr.Zero);
CvInvoke.cvPow(outReal, outReal, 2.0);
CvInvoke.cvPow(outIm, outIm, 2.0);
CvInvoke.cvAdd(outReal, outIm, outReal, IntPtr.Zero);
CvInvoke.cvPow(outReal, outReal, 0.5);
CvInvoke.cvAddS(outReal, new MCvScalar(1.0), outReal, IntPtr.Zero); // 1 + Mag
CvInvoke.cvLog(outReal, outReal); // log(1 + Mag)
return outReal;
}
// We have to switch quadrants so that the origin is at the image center
private void SwitchQuadrants(ref Matrix<float> matrix)
{
int cx = matrix.Cols / 2;
int cy = matrix.Rows / 2;
Matrix<float> q0 = matrix.GetSubRect(new Rectangle(0, 0, cx, cy));
Matrix<float> q1 = matrix.GetSubRect(new Rectangle(cx, 0, cx, cy));
Matrix<float> q2 = matrix.GetSubRect(new Rectangle(0, cy, cx, cy));
Matrix<float> q3 = matrix.GetSubRect(new Rectangle(cx, cy, cx, cy));
Matrix<float> tmp = new Matrix<float>(q0.Size);
q0.CopyTo(tmp);
q3.CopyTo(q0);
tmp.CopyTo(q3);
q1.CopyTo(tmp);
q2.CopyTo(q1);
tmp.CopyTo(q2);
}
Most of the information in this answer is from a question on the OpenCV mailing list and Steve Eddins' article on FFT in image processing.
Related
I want to rotate a Bitmap around a given point and have that point become the new center of the Bitmap.
Here is what I tried first
Bitmap rotate(Bitmap img, float angle, int cx, int cy)
{
Bitmap result = new Bitmap(img.Width, img.Height);
int middleX = img.Width / 2,
middleY = img.Height / 2;
using (Graphics g = Graphics.FromImage(result))
{
g.Clear(Color.Black);
g.TranslateTransform(cx, cy);
g.RotateTransform(angle);
g.TranslateTransform(-cx, -cy);
g.TranslateTransform(middleX - cx, middleY - cy); //shift (cx, cy) to be at the center, does not work
g.DrawImage(originalImage, new Point(0, 0));
}
return result;
}
But when I go to translate the image after rotation, the translation is in the original space instead of this new rotated space and doesn't come out right.
I tried basically every combination of things I could think of with no luck. Search results only describe how to rotate around a point.
Original image (red dot is the point to rotate around) -
After rotating by 45 degrees the image should be translated so that the red dot is the center of the image
You have to set the matrix order to Append for the final translation. I have no idea why this is what makes it work. I literally just brute force tried every possible way of doing everything until it worked. I'd still be interested in an explanation of what is going on though.
Bitmap rotate(Bitmap img, float angle, int cx, int cy)
{
Bitmap result = new Bitmap(img.Width, img.Height);
int mx = img.Width / 2,
my = img.Height / 2;
using (Graphics g = Graphics.FromImage(result))
{
g.Clear(Color.Black);
g.TranslateTransform(cx, cy);
g.RotateTransform(angle);
g.TranslateTransform(-cx, -cy);
g.TranslateTransform(mx - cx, my - cy, MatrixOrder.Append);
g.DrawImage(originalImage, new Point(0, 0));
}
return result;
}
Im undistorting an image using intrinsic values of camera martix and distortion coeeficient. It corrects the distortion perfectly. But if I load same image twice, sometimes it gives out a blank gray image. Not sure why this is happening.
void distort()
{
Mat cameraMatrix = new Mat(3, 3, DepthType.Cv64F,1);
Mat distCoeffs = new Mat(8, 1, DepthType.Cv64F, 1);
//Camera matrix values
double fx = 5715.63;
double fy = 5788.09;
double cx = 1339.66;
double cy = 941.89;
//double skew = 0.11273;
//Optical distortion coefficient values
double k1 = -2.006166;
double k2 = 5.478401;
//double k3 = 0.001513;
double p1 = 0.007517;
double p2 = -0.007439;
//Set the camera matrix
cameraMatrix.SetValue(0, 0, fx);
cameraMatrix.SetValue(1, 1, fy);
cameraMatrix.SetValue(1, 0, 1);
cameraMatrix.SetValue(0, 2, cx);
cameraMatrix.SetValue(1, 2, cy);
cameraMatrix.SetValue(2, 2, 1);
//Set the distortion matrix
distCoeffs.SetValue(0, 0, k1);
distCoeffs.SetValue(1, 0, k2);
distCoeffs.SetValue(2, 0, p1);
distCoeffs.SetValue(3, 0, p2);
//Read the input image
image = CvInvoke.Imread(#"33.bmp", ImreadModes.Color);
Correct the input image
Mat outFrame = image.Clone();
CvInvoke.Undistort(image, outFrame, cameraMatrix, distCoeffs);
//save the corrected image
image = outFrame.Clone();
imageBox1.Image = image.Bitmap;
image.Dispose();
outFrame.Dispose();
}
I am new to Emgucv and I am working on face movement direction detection
I had come across many codes in Internet but they are very difficult to understand.
So can you please provide a easy and understandable code or links for learning this situation.
Thanks in advance
Based on this :
Bitmap bmp = new Bitmap(); // your bitmap contain a face
Mat mat = GetMatFromSDImage(bmp);
using (var nextFrame = mat.ToImage<Bgr, Byte>())
{
if (nextFrame != null)
{
Image<Gray, byte> grayframe = nextFrame.Convert<Gray, byte>();
Rectangle[] faces = mHaarCascade.DetectMultiScale(grayframe, 1.1, 10, Size.Empty);
if (faces.Count() > 0)
{
// some faces are detected
// you can check the X and Y of faces here
}
}
}
private Mat GetMatFromSDImage(Bitmap image)
{
int stride = 0;
System.Drawing.Rectangle rect = new System.Drawing.Rectangle(0, 0, image.Width, image.Height);
System.Drawing.Imaging.BitmapData bmpData = image.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite, image.PixelFormat);
System.Drawing.Imaging.PixelFormat pf = image.PixelFormat;
if (pf == System.Drawing.Imaging.PixelFormat.Format32bppArgb)
{
stride = image.Width * 4;
}
else
{
stride = image.Width * 3;
}
Image<Bgra, byte> cvImage = new Image<Bgra, byte>(image.Width, image.Height, stride, (IntPtr)bmpData.Scan0);
image.UnlockBits(bmpData);
return cvImage.Mat;
}
so faces contains the array of Rectangle bounding faces. you can check the X and Y property of the rectangles to check if it moves and compares it with the initial position to detect it's direction.
Update based on comment
To detect head rotation a simple solution can be eye detection. you can use haarcascade_eye.xml to detect ayes. then you can calculate the rotation form each eye's X and Y position.
Here you can find a simple example of eye detection
The following code behaves very oddly. It adds some extra spacing at the bottom of the image, and I can not see why. Result of code:
And this is the code that I am working with:
public static Image ReDraw(this Image main, int w, int h,
CompositingQuality quality = CompositingQuality.Default, //linear?
SmoothingMode smoothing_mode = SmoothingMode.None,
InterpolationMode ip_mode = InterpolationMode.NearestNeighbor)
{
//size
double dbl = (double)main.Width / (double)main.Height;
//preserve size ratio
if ((int)((double)h * dbl) <= w)
w = (int)((double)h * dbl);
else
h = (int)((double)w / dbl);
//draw
Image newImage = new System.Drawing.Bitmap(w, h);
Graphics thumbGraph = Graphics.FromImage(newImage);
thumbGraph.CompositingQuality = quality;
thumbGraph.SmoothingMode = smoothing_mode;
thumbGraph.InterpolationMode = ip_mode;
thumbGraph.Clear(Color.Transparent);
thumbGraph.DrawImage(main, 0, 0, w, h);
thumbGraph.DrawImage(main,
new System.Drawing.Rectangle(0, 0, w, h),
new System.Drawing.Rectangle(0, 0, main.Width, main.Height),
System.Drawing.GraphicsUnit.Pixel);
return newImage;
}
thumbGraph.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.HighQuality;
I'm trying to visualize the FFT of an image with EMGU. Here's the image I'm processing:
Here's the expected result:
Here's what I get:
Here's my code:
Image<Gray, float> image = new Image<Gray, float>(#"C:\Users\me\Desktop\sample3.jpg");
IntPtr complexImage = CvInvoke.cvCreateImage(image.Size, Emgu.CV.CvEnum.IPL_DEPTH.IPL_DEPTH_32F, 2);
CvInvoke.cvSetZero(complexImage);
CvInvoke.cvSetImageCOI(complexImage, 1);
CvInvoke.cvCopy(image, complexImage, IntPtr.Zero);
CvInvoke.cvSetImageCOI(complexImage, 0);
Matrix<float> dft = new Matrix<float>(image.Rows, image.Cols, 2);
CvInvoke.cvDFT(complexImage, dft, Emgu.CV.CvEnum.CV_DXT.CV_DXT_FORWARD, 0);
Matrix<float> outReal = new Matrix<float>(image.Size);
Matrix<float> outIm = new Matrix<float>(image.Size);
CvInvoke.cvSplit(dft, outReal, outIm, IntPtr.Zero, IntPtr.Zero);
Image<Gray, float> fftImage = new Image<Gray, float>(outReal.Size);
CvInvoke.cvCopy(outReal, fftImage, IntPtr.Zero);
pictureBox1.Image = image.ToBitmap();
pictureBox2.Image = fftImage.Log().ToBitmap();
What mistake am I making here?
Update: as per Roger Rowland's suggestion here's my updated code. The result looks better but I'm not 100% sure it's correct. Here's the result:
Image<Gray, float> image = new Image<Gray, float>(#"C:\Users\yytov\Desktop\sample3.jpg");
IntPtr complexImage = CvInvoke.cvCreateImage(image.Size, Emgu.CV.CvEnum.IPL_DEPTH.IPL_DEPTH_32F, 2);
CvInvoke.cvSetZero(complexImage); // Initialize all elements to Zero
CvInvoke.cvSetImageCOI(complexImage, 1);
CvInvoke.cvCopy(image, complexImage, IntPtr.Zero);
CvInvoke.cvSetImageCOI(complexImage, 0);
Matrix<float> dft = new Matrix<float>(image.Rows, image.Cols, 2);
CvInvoke.cvDFT(complexImage, dft, Emgu.CV.CvEnum.CV_DXT.CV_DXT_FORWARD, 0);
//The Real part of the Fourier Transform
Matrix<float> outReal = new Matrix<float>(image.Size);
//The imaginary part of the Fourier Transform
Matrix<float> outIm = new Matrix<float>(image.Size);
CvInvoke.cvSplit(dft, outReal, outIm, IntPtr.Zero, IntPtr.Zero);
CvInvoke.cvPow(outReal, outReal, 2.0);
CvInvoke.cvPow(outIm, outIm, 2.0);
CvInvoke.cvAdd(outReal, outIm, outReal, IntPtr.Zero);
CvInvoke.cvPow(outReal, outReal, 0.5);
CvInvoke.cvAddS(outReal, new MCvScalar(1.0), outReal, IntPtr.Zero); // 1 + Mag
CvInvoke.cvLog(outReal, outReal); // log(1 + Mag)
// Swap quadrants
int cx = outReal.Cols / 2;
int cy = outReal.Rows / 2;
Matrix<float> q0 = outReal.GetSubRect(new Rectangle(0, 0, cx, cy));
Matrix<float> q1 = outReal.GetSubRect(new Rectangle(cx, 0, cx, cy));
Matrix<float> q2 = outReal.GetSubRect(new Rectangle(0, cy, cx, cy));
Matrix<float> q3 = outReal.GetSubRect(new Rectangle(cx, cy, cx, cy));
Matrix<float> tmp = new Matrix<float>(q0.Size);
q0.CopyTo(tmp);
q3.CopyTo(q0);
tmp.CopyTo(q3);
q1.CopyTo(tmp);
q2.CopyTo(q1);
tmp.CopyTo(q2);
CvInvoke.cvNormalize(outReal, outReal, 0.0, 255.0, Emgu.CV.CvEnum.NORM_TYPE.CV_MINMAX, IntPtr.Zero);
Image<Gray, float> fftImage = new Image<Gray, float>(outReal.Size);
CvInvoke.cvCopy(outReal, fftImage, IntPtr.Zero);
pictureBox1.Image = image.ToBitmap();
pictureBox2.Image = fftImage.ToBitmap();
I cannot comment on the magnitude/intensity of the resulting image, but I can give you a tip about the spatial distribution of points in your image.
OpenCV doesn't rearrange the quadrants to put origin [0,0] into center of image. You have to rearrange the quadrants manually.
Look at step 6 at the following page:
http://docs.opencv.org/doc/tutorials/core/discrete_fourier_transform/discrete_fourier_transform.html
It's official doc for OpenCV, so it's in C++, but principle holds.