In this research article, in the section 4.3.1 (core detection),
How can I calculate correlation coefficients between two pixels?
public static Complex[,] Correlation(Complex[,]image, Complex[,]mask)
{
Complex[,] convolve = null;
int imageWidth = image.GetLength(0);
int imageHeight = image.GetLength(1);
int maskWidth = mask.GetLength(0);
int maskeHeight = mask.GetLength(1);
if (imageWidth == maskWidth && imageHeight == maskeHeight)
{
FourierTransform ftForImage = new FourierTransform(image); ftForImage.ForwardFFT();
FourierTransform ftForMask = new FourierTransform(mask); ftForMask.ForwardFFT();
Complex[,] fftImage = ftForImage.FourierImageComplex;
Complex[,] fftKernel = ftForMask.FourierImageComplex;
Complex[,] fftConvolved = new Complex[imageWidth, imageHeight];
for (int j = 0; j < imageHeight; j++)
{
for (int i = 0; i < imageWidth; i++)
{
fftConvolved[i,j] = Complex.Conjugate(fftImage[i,j]) * fftKernel[i,j];
}
}
FourierTransform ftForConv = new FourierTransform();
ftForConv.InverseFFT(fftConvolved);
convolve = ftForConv.GrayscaleImageComplex;
Rescale(convolve);
convolve = FourierShifter.FFTShift(convolve);
}
else
{
throw new Exception("padding needed");
}
return convolve;
}
Is this the correct procedure to calculate correlations?
If yes, how can I find Correlation-coefficients from that?
In article correlation are calculated between two "windows", i.e. between two sets of points, not between two points. if i'm not mistaken, correlation coefficient is a scalar value, not a vector. In signal processing , correlation calculated as sum of multiplications divided by sum of squares of signal values. It may be incorrect in details, but in general, correlation calculated like this:
correlation = sum(S1[i]*S2[i])/sqrt(sum(S1[i]^2 * S2[i]^2));
For 2-dimention case (window) just add second index:
correlation = sum(S1[i,j]*S2[i,j])/sqrt(sum(S1[i,j]^2 * S2[i,j]^2));
Related
Take a look at this link.
public struct COMPLEX
{
public double real, imag;
public Complex(double x, double y)
{
real = x;
imag = y;
}
public float Magnitude()
{
return ((float)Math.Sqrt(real * real + imag * imag));
}
public float Phase()
{
return ((float)Math.Atan(imag / real));
}
}
public static COMPLEX[,] ApplyFilterHMMFreqDomain(COMPLEX[,] FFTData,
float rH, float rL, float Sigma, float Slope)
{
COMPLEX[,] Output = new COMPLEX[FFTData.GetLength(0), FFTData.GetLength(1)];
int i, j, W, H;
W = FFTData.GetLength(0);
H = FFTData.GetLength(1);
double Weight;
//Taking FFT of Gaussian HPF
double[,] GaussianHPF =
GenerateGaussianKernelHPF(FFTData.GetLength(0), Sigma, Slope, out Weight);
//Variables for FFT of Gaussian Filter
COMPLEX[,] GaussianHPFFFT;
for (i = 0; i <= GaussianHPF.GetLength(0) - 1; i++)
for (j = 0; j <= GaussianHPF.GetLength(1) - 1; j++)
{
GaussianHPF[i, j] = GaussianHPF[i, j];// / Weight;
}
FFT GaussianFFTObject = new FFT(GaussianHPF);
GaussianFFTObject.ForwardFFT(GaussianHPF);
//Shifting FFT for Filtering
GaussianFFTObject.FFTShift();
GaussianHPFFFT = GaussianFFTObject.FFTShifted;
for (i = 0; i <= GaussianHPF.GetLength(0) - 1; i++)
for (j = 0; j <= GaussianHPF.GetLength(1) - 1; j++)
{
GaussianHPFFFT[i, j].real = (rH - rL) * GaussianHPFFFT[i, j].real + rL;
GaussianHPFFFT[i, j].imag = (rH - rL) * GaussianHPFFFT[i, j].imag + rL;
}
// Applying Filter on the FFT of the Log Image by Multiplying in Frequency Domain
Output = MultiplyFFTMatrices(GaussianHPFFFT, FFTData);
return Output;
}
After detail experiment, I found no or insignificant effect of the following snippet of source code:
for (i = 0; i <= GaussianHPF.GetLength(0) - 1; i++)
for (j = 0; j <= GaussianHPF.GetLength(1) - 1; j++)
{
GaussianHPFFFT[i, j].real = (rH - rL) * GaussianHPFFFT[i, j].real + rL;
GaussianHPFFFT[i, j].imag = (rH - rL) * GaussianHPFFFT[i, j].imag + rL;
}
Can anyone explain why this part of the code is/isn't important?
That snippet of code applies the same linear transformation to each value of the frequency domain. This is equivalent to applying the same transformation to each pixel in the spatial domain (the Fourier transform is linear)
The linear transformation and the convolution commute, as is easy to see since the convolution is a multiplication in the frequency domain, you can clearly swap these two operations there.
Thus, you are applying this constant, linear transformation to the output of the filter. If you display the output by linearly scaling the values to the output range, you will undo any effect of this transformation.
My guess is that it’s there to scale the output spatial-domain image to a meaningful range.
I am trying to replicate the outcome of this link using linear convolution in spatial-domain.
Images are first converted to 2d double arrays and then convolved. Image and kernel are of the same size. The image is padded before convolution and cropped accordingly after the convolution.
As compared to the FFT-based convolution, the output is weird and incorrect.
How can I solve the issue?
Note that I obtained the following image output from Matlab which matches my C# FFT output:
.
Update-1: Following #Ben Voigt's comment, I changed the Rescale() function to replace 255.0 with 1 and thus the output is improved substantially. But, still, the output doesn't match the FFT output (which is the correct one).
.
Update-2: Following #Cris Luengo's comment, I have padded the image by stitching and then performed spatial convolution. The outcome has been as follows:
So, the output is worse than the previous one. But, this has a similarity with the 2nd output of the linked answer which means a circular convolution is not the solution.
.
Update-3: I have used the Sum() function proposed by #Cris Luengo's answer. The result is a more improved version of **Update-1**:
But, it is still not 100% similar to the FFT version.
.
Update-4: Following #Cris Luengo's comment, I have subtracted the two outcomes to see the difference:
,
1. spatial minus frequency domain
2. frequency minus spatial domain
Looks like, the difference is substantial which means, spatial convolution is not being done correctly.
.
Source Code:
(Notify me if you need more source code to see.)
public static double[,] LinearConvolutionSpatial(double[,] image, double[,] mask)
{
int maskWidth = mask.GetLength(0);
int maskHeight = mask.GetLength(1);
double[,] paddedImage = ImagePadder.Pad(image, maskWidth);
double[,] conv = Convolution.ConvolutionSpatial(paddedImage, mask);
int cropSize = (maskWidth/2);
double[,] cropped = ImageCropper.Crop(conv, cropSize);
return conv;
}
static double[,] ConvolutionSpatial(double[,] paddedImage1, double[,] mask1)
{
int imageWidth = paddedImage1.GetLength(0);
int imageHeight = paddedImage1.GetLength(1);
int maskWidth = mask1.GetLength(0);
int maskHeight = mask1.GetLength(1);
int convWidth = imageWidth - ((maskWidth / 2) * 2);
int convHeight = imageHeight - ((maskHeight / 2) * 2);
double[,] convolve = new double[convWidth, convHeight];
for (int y = 0; y < convHeight; y++)
{
for (int x = 0; x < convWidth; x++)
{
int startX = x;
int startY = y;
convolve[x, y] = Sum(paddedImage1, mask1, startX, startY);
}
}
Rescale(convolve);
return convolve;
}
static double Sum(double[,] paddedImage1, double[,] mask1, int startX, int startY)
{
double sum = 0;
int maskWidth = mask1.GetLength(0);
int maskHeight = mask1.GetLength(1);
for (int y = startY; y < (startY + maskHeight); y++)
{
for (int x = startX; x < (startX + maskWidth); x++)
{
double img = paddedImage1[x, y];
double msk = mask1[x - startX, y - startY];
sum = sum + (img * msk);
}
}
return sum;
}
static void Rescale(double[,] convolve)
{
int imageWidth = convolve.GetLength(0);
int imageHeight = convolve.GetLength(1);
double maxAmp = 0.0;
for (int j = 0; j < imageHeight; j++)
{
for (int i = 0; i < imageWidth; i++)
{
maxAmp = Math.Max(maxAmp, convolve[i, j]);
}
}
double scale = 1.0 / maxAmp;
for (int j = 0; j < imageHeight; j++)
{
for (int i = 0; i < imageWidth; i++)
{
double d = convolve[i, j] * scale;
convolve[i, j] = d;
}
}
}
public static Bitmap ConvolveInFrequencyDomain(Bitmap image1, Bitmap kernel1)
{
Bitmap outcome = null;
Bitmap image = (Bitmap)image1.Clone();
Bitmap kernel = (Bitmap)kernel1.Clone();
//linear convolution: sum.
//circular convolution: max
uint paddedWidth = Tools.ToNextPow2((uint)(image.Width + kernel.Width));
uint paddedHeight = Tools.ToNextPow2((uint)(image.Height + kernel.Height));
Bitmap paddedImage = ImagePadder.Pad(image, (int)paddedWidth, (int)paddedHeight);
Bitmap paddedKernel = ImagePadder.Pad(kernel, (int)paddedWidth, (int)paddedHeight);
Complex[,] cpxImage = ImageDataConverter.ToComplex(paddedImage);
Complex[,] cpxKernel = ImageDataConverter.ToComplex(paddedKernel);
// call the complex function
Complex[,] convolve = Convolve(cpxImage, cpxKernel);
outcome = ImageDataConverter.ToBitmap(convolve);
outcome = ImageCropper.Crop(outcome, (kernel.Width/2)+1);
return outcome;
}
Your current output looks more like the auto-correlation function than the convolution of Lena with herself. I think the issue might be in your Sum function.
If you look at the definition of the convolution sum, you'll see that the kernel (or the image, doesn't matter) is mirrored:
sum_m( f[n-m] g[m] )
For the one function, m appears with a plus sign, and for the other it appears with a minus sign.
You'll need to modify your Sum function to read the mask1 image in the right order:
static double Sum(double[,] paddedImage1, double[,] mask1, int startX, int startY)
{
double sum = 0;
int maskWidth = mask1.GetLength(0);
int maskHeight = mask1.GetLength(1);
for (int y = startY; y < (startY + maskHeight); y++)
{
for (int x = startX; x < (startX + maskWidth); x++)
{
double img = paddedImage1[x, y];
double msk = mask1[maskWidth - x + startX - 1, maskHeight - y + startY - 1];
sum = sum + (img * msk);
}
}
return sum;
}
The other option is to pass a mirrored version of mask1 to this function.
I have found the solution from this link. The main clue was to introduce an offset and a factor.
factor is the sum of all values in the kernel.
offset is an arbitrary value to fix the output further.
.
#Cris Luengo's answer also raised a valid point.
.
The following source code is supplied in the given link:
private void SafeImageConvolution(Bitmap image, ConvMatrix fmat)
{
//Avoid division by 0
if (fmat.Factor == 0)
return;
Bitmap srcImage = (Bitmap)image.Clone();
int x, y, filterx, filtery;
int s = fmat.Size / 2;
int r, g, b;
Color tempPix;
for (y = s; y < srcImage.Height - s; y++)
{
for (x = s; x < srcImage.Width - s; x++)
{
r = g = b = 0;
// Convolution
for (filtery = 0; filtery < fmat.Size; filtery++)
{
for (filterx = 0; filterx < fmat.Size; filterx++)
{
tempPix = srcImage.GetPixel(x + filterx - s, y + filtery - s);
r += fmat.Matrix[filtery, filterx] * tempPix.R;
g += fmat.Matrix[filtery, filterx] * tempPix.G;
b += fmat.Matrix[filtery, filterx] * tempPix.B;
}
}
r = Math.Min(Math.Max((r / fmat.Factor) + fmat.Offset, 0), 255);
g = Math.Min(Math.Max((g / fmat.Factor) + fmat.Offset, 0), 255);
b = Math.Min(Math.Max((b / fmat.Factor) + fmat.Offset, 0), 255);
image.SetPixel(x, y, Color.FromArgb(r, g, b));
}
}
}
I am using the following Kernel,
double[,] kernel = new double[,] { { -1, -1, -1, },
{ -1, 9, -1, },
{ -1, -1, -1, }, };
The following code seems to be blurring the input image, rather than Sharpening.
What could have been the issue here?
Here is the entire VS2013 solution.
The original image,
The resulting blurred image,
I have written the following code to Sharpen an image,
public static Bitmap FftSharpen(Bitmap image, double [,] mask)
{
if (image.PixelFormat == PixelFormat.Format8bppIndexed)
{
Bitmap imageClone = (Bitmap)image.Clone();
double[,] maskClone = (double[,])mask.Clone();
Complex[,] cPaddedImage = ImageDataConverter.ToComplex(imageClone);
Complex[,] cPaddedMask = ImageDataConverter.ToComplex(maskClone);
Complex[,] cConvolved = Convolution.Convolve(cPaddedImage, cPaddedMask);
return ImageDataConverter.ToBitmap(cConvolved);
}
else
{
throw new Exception("not a grascale");
}
}
.
.
P.S.
The following is my convolution code,
public static class Convolution
{
public static Complex[,] Convolve(Complex[,] image, Complex[,] mask)
{
Complex[,] convolve = null;
int imageWidth = image.GetLength(0);
int imageHeight = image.GetLength(1);
int maskWidth = mask.GetLength(0);
int maskeHeight = mask.GetLength(1);
if (imageWidth == maskWidth && imageHeight == maskeHeight)
{
FourierTransform ftForImage = new FourierTransform(image); ftForImage.ForwardFFT();
FourierTransform ftForMask = new FourierTransform(mask); ftForMask.ForwardFFT();
Complex[,] fftImage = ftForImage.FourierImageComplex;
Complex[,] fftKernel = ftForMask.FourierImageComplex;
Complex[,] fftConvolved = new Complex[imageWidth, imageHeight];
for (int j = 0; j < imageHeight; j++)
{
for (int i = 0; i < imageWidth; i++)
{
fftConvolved[i, j] = fftImage[i, j] * fftKernel[i, j];
}
}
FourierTransform ftForConv = new FourierTransform();
ftForConv.InverseFFT(fftConvolved);
convolve = ftForConv.GrayscaleImageComplex;
Rescale(convolve);
convolve = FourierShifter.FFTShift(convolve);
}
else
{
throw new Exception("padding needed");
}
return convolve;
}
//Rescale values between 0 and 255.
private static void Rescale(Complex[,] convolve)
{
int imageWidth = convolve.GetLength(0);
int imageHeight = convolve.GetLength(1);
double maxAmp = 0.0;
for (int j = 0; j < imageHeight; j++)
{
for (int i = 0; i < imageWidth; i++)
{
maxAmp = Math.Max(maxAmp, convolve[i, j].Magnitude);
}
}
double scale = 255.0 / maxAmp;
for (int j = 0; j < imageHeight; j++)
{
for (int i = 0; i < imageWidth; i++)
{
convolve[i, j] = new Complex(convolve[i, j].Real * scale, convolve[i, j].Imaginary * scale);
maxAmp = Math.Max(maxAmp, convolve[i, j].Magnitude);
}
}
}
}
Similarly to your other question, the kernel is obtained from an unsigned Bitmap which result in the effective kernel
255 255 255
255 9 255
255 255 255
instead of the expected
-1 -1 -1
-1 9 -1
-1 -1 -1
A solution would again be to convert the bitmap to signed values. Alternatively, since the code provided in this question also supports the numerical kernel to be provided directly to FftSharpen, you could pad _numericalKernel with:
public class MatrixPadder
{
public static double[,] Pad(double[,] image, int newWidth, int newHeight)
{
int width = image.GetLength(0);
int height = image.GetLength(1);
/*
It is always guaranteed that,
width < newWidth
and
height < newHeight
*/
if ((width < newWidth && height < newHeight)
|| (width < newWidth && height == newHeight)
|| (width == newWidth && height < newHeight))
{
double[,] paddedImage = new double[newWidth, newHeight];
int startPointX = (int)Math.Ceiling((double)(newWidth - width) / (double)2) - 1;
int startPointY = (int)Math.Ceiling((double)(newHeight - height) / (double)2) - 1;
for (int y = startPointY; y < (startPointY + height); y++)
{
for (int x = startPointX; x < (startPointX + width); x++)
{
int xxx = x - startPointX;
int yyy = y - startPointY;
paddedImage[x, y] = image[xxx, yyy];
}
}
return paddedImage;
}
else if (width == newWidth && height == newHeight)
{
return image;
}
else
{
throw new Exception("Pad() -- threw an exception");
}
}
}
which you could call from filterButton_Click using:
if (_convolutionType == ConvolutionType.FFT)
{
double[,] paddedmask = MatrixPadder.Pad(_numericalKernel,
_paddedImage.Width,
_paddedImage.Height);
sharpened = SharpenFilter.FftSharpen(_paddedImage, paddedmask);
}
Also adjusting the Rescale function as shown in my other answer should then give you the desired sharpened image:
I want to Convolve Lena with itself in the Frequency Domain. Here is an excerpt from a book. which suggests how should the output of the convolution be:
I have written the following application to achieve the Convolution of two images in the frequency domain. The steps I followed are as follows:
Convert Lena into a matrix of complex numbers.
Apply FFT to obtain a complex matrix.
Multiply two complex matrices element by element (if that is the definition of Convolution).
Apply IFFT to the result of the multiplication.
The output seems to be not coming as expected:
Two issues are visible here:
The output only contains a black background with only one dot at its center.
The original image gets distorted after the the execution of convolution.
.
Note. FFT and I-FFT are working perfectly with the same libraries.
Note-2. There is a thread in SO that seems to be discussing the same topic.
.
Source Code:
public static class Convolution
{
public static Complex[,] Convolve(Complex[,]image, Complex[,]mask)
{
Complex[,] convolve = null;
int imageWidth = image.GetLength(0);
int imageHeight = image.GetLength(1);
int maskWidth = mask.GetLength(0);
int maskeHeight = mask.GetLength(1);
if (imageWidth == maskWidth && imageHeight == maskeHeight)
{
FourierTransform ftForImage = new FourierTransform(image); ftForImage.ForwardFFT();
FourierTransform ftForMask = new FourierTransform(mask); ftForMask.ForwardFFT();
Complex[,] fftImage = ftForImage.FourierTransformedImageComplex;
Complex[,] fftKernel = ftForMask.FourierTransformedImageComplex;
Complex[,] fftConvolved = new Complex[imageWidth, imageHeight];
for (int i = 0; i < imageWidth; i++)
{
for (int j = 0; j < imageHeight; j++)
{
fftConvolved[i, j] = fftImage[i, j] * fftKernel[i, j];
}
}
FourierTransform ftForConv = new FourierTransform();
ftForConv.InverseFFT(fftConvolved);
convolve = ftForConv.GrayscaleImageComplex;
//convolve = fftConvolved;
}
else
{
throw new Exception("padding needed");
}
return convolve;
}
}
private void convolveButton_Click(object sender, EventArgs e)
{
Bitmap lena = inputImagePictureBox.Image as Bitmap;
Bitmap paddedMask = paddedMaskPictureBox.Image as Bitmap;
Complex[,] cLena = ImageDataConverter.ToComplex(lena);
Complex[,] cPaddedMask = ImageDataConverter.ToComplex(paddedMask);
Complex[,] cConvolved = Convolution.Convolve(cLena, cPaddedMask);
Bitmap convolved = ImageDataConverter.ToBitmap(cConvolved);
convolvedImagePictureBox.Image = convolved;
}
There is a difference in how you call InverseFFT between the working FFT->IFFT application, and the broken Convolution application. In the latter case you do not pass explicitly the Width and Height parameters (which you are supposed to get from the input image):
public void InverseFFT(Complex[,] fftImage)
{
if (FourierTransformedImageComplex == null)
{
FourierTransformedImageComplex = fftImage;
}
GrayscaleImageComplex = FourierFunction.FFT2D(FourierTransformedImageComplex, Width, Height, -1);
GrayscaleImageInteger = ImageDataConverter.ToInteger(GrayscaleImageComplex);
InputImageBitmap = ImageDataConverter.ToBitmap(GrayscaleImageInteger);
}
As a result both Width and Height are 0 and the code skips over most of the inverse 2D transformation. Initializing those parameters should give you something which is at least not all black.
if (FourierTransformedImageComplex == null)
{
FourierTransformedImageComplex = fftImage;
Width = fftImage.GetLength(0);
Height = fftImage.GetLength(1);
}
Then you should notice some sharp white/black edges. Those are caused by wraparounds in the output values. To avoid this, you may want to rescale the output after the inverse transform to fit the available scale with something such as:
double maxAmp = 0.0;
for (int i = 0; i < imageWidth; i++)
{
for (int j = 0; j < imageHeight; j++)
{
maxAmp = Math.Max(maxAmp, convolve[i, j].Magnitude);
}
}
double scale = 255.0 / maxAmp;
for (int i = 0; i < imageWidth; i++)
{
for (int j = 0; j < imageHeight; j++)
{
convolve[i, j] = new Complex(convolve[i, j].Real * scale, convolve[i, j].Imaginary * scale);
maxAmp = Math.Max(maxAmp, convolve[i, j].Magnitude);
}
}
This should then give the more reasonable output:
However that is still not as depicted in your book. At this point we have a 2D circular convolution. To get a 2D linear convolution, you need to make sure the images are both padded to the sum of the dimensions:
Bitmap lena = inputImagePictureBox.Image as Bitmap;
Bitmap mask = paddedMaskPictureBox.Image as Bitmap;
Bitmap paddedLena = ImagePadder.Pad(lena, lena.Width+ mask.Width, lena.Height+ mask.Height);
Bitmap paddedMask = ImagePadder.Pad(mask, lena.Width+ mask.Width, lena.Height+ mask.Height);
Complex[,] cLena = ImageDataConverter.ToComplex(paddedLena);
Complex[,] cPaddedMask = ImageDataConverter.ToComplex(paddedMask);
Complex[,] cConvolved = Convolution.Convolve(cLena, cPaddedMask);
And as you adjust the padding, you may want to change the padding color to black otherwise your padding will in itself introduce a large correlation between the two images:
public class ImagePadder
{
public static Bitmap Pad(Bitmap maskImage, int newWidth, int newHeight)
{
...
Grayscale.Fill(resizedImage, Color.Black);
Now you should be getting the following:
We are getting close, but the peak of the autocorrelation result is not in the center, and that's because you FourierShifter.FFTShift in the forward transform but do not call the corresponding FourierShifter.RemoveFFTShift in the inverse transform. If we adjust those (either remove FFTShift in ForwardFFT, or add RemoveFFTShift in InverseFFT), then we finally get:
I've got torubles with appling gaussian blur to image in frequency domain.
For unknown reasons (probably I've dont something wrong) I recieve wired image instead of blurred one.
There's what i do step by step:
Load the image.
Split image into separate channels.
private static Bitmap[] separateColorChannels(Bitmap source, int channelCount)
{
if (channelCount != 3 && channelCount != 4)
{
throw new NotSupportedException("Bitmap[] FFTServices.separateColorChannels(Bitmap, int): Only 3 and 4 channels are supported.");
}
Bitmap[] result = new Bitmap[channelCount];
LockBitmap[] locks = new LockBitmap[channelCount];
LockBitmap sourceLock = new LockBitmap(source);
sourceLock.LockBits();
for (int i = 0; i < channelCount; ++i)
{
result[i] = new Bitmap(source.Width, source.Height, PixelFormat.Format8bppIndexed);
locks[i] = new LockBitmap(result[i]);
locks[i].LockBits();
}
for (int x = 0; x < source.Width; x++)
{
for (int y = 0; y < source.Height; y++)
{
switch (channelCount)
{
case 3:
locks[0].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).R));
locks[1].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).G));
locks[2].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).B));
break;
case 4:
locks[0].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).A));
locks[1].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).R));
locks[2].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).G));
locks[3].SetPixel(x, y, Color.FromArgb(sourceLock.GetPixel(x, y).B));
break;
default:
break;
}
}
}
for (int i = 0; i < channelCount; ++i)
{
locks[i].UnlockBits();
}
sourceLock.UnlockBits();
}
Convert every channel into complex images (with AForge.NET).
public static AForge.Imaging.ComplexImage[] convertColorChannelsToComplex(Emgu.CV.Image<Emgu.CV.Structure.Gray, Byte>[] channels)
{
AForge.Imaging.ComplexImage[] result = new AForge.Imaging.ComplexImage[channels.Length];
for (int i = 0; i < channels.Length; ++i)
{
result[i] = AForge.Imaging.ComplexImage.FromBitmap(channels[i].Bitmap);
}
return result;
}
Apply Gaussian blur.
First i create the kernel (For testing purposes kernel size is equal to image size, tho only center part of it is calculated with gaussian function, rest of kernel is equal to re=1 im=0).
private ComplexImage makeGaussKernel(int side, double min, double max, double step, double std)
{
// get value at top left corner
double _0x0 = gauss2d(min, min, std);
// top left corner should be 1, so making scaler for rest of the values
double scaler = 1 / _0x0;
int pow2 = SizeServices.getNextNearestPowerOf2(side);
Bitmap bitmap = new Bitmap(pow2, pow2, PixelFormat.Format8bppIndexed);
var result = AForge.Imaging.ComplexImage.FromBitmap(bitmap);
// For test purposes my kernel is size of image, so first, filling with 1 only.
for (int i = 0; i < result.Data.GetLength(0); ++i)
{
for (int j = 0; j < result.Data.GetLength(0); ++j)
{
result.Data[i, j].Re = 1;
result.Data[i, j].Im = 0;
}
}
// The real kernel's size.
int count = (int)((Math.Abs(max) + Math.Abs(min)) / step);
double h = min;
// Calculating kernel's values and storing them somewhere in the center of kernel.
for (int i = result.Data.GetLength(0) / 2 - count / 2; i < result.Data.GetLength(0) / 2 + count / 2; ++i)
{
double w = min;
for (int j = result.Data.GetLength(1) / 2 - count / 2; j < result.Data.GetLength(1) / 2 + count / 2; ++j)
{
result.Data[i, j].Re = (scaler * gauss2d(w, h, std)) * 255;
w += step;
}
h += step;
}
return result;
}
// The gauss function
private double gauss2d(double x, double y, double std)
{
return ((1.0 / (2 * Math.PI * std * std)) * Math.Exp(-((x * x + y * y) / (2 * std * std))));
}
Apply FFT to every channel and kernel.
Multiply center part of every channel by kernel.
void applyFilter(/*shortened*/)
{
// Image's size is 512x512 that's why 512 is hardcoded here
// min = -2.0; max = 2.0; step = 0.33; std = 11
ComplexImage filter = makeGaussKernel(512, min, max, step, std);
// Applies FFT (with AForge.NET) to every channel and filter
applyFFT(complexImage);
applyFFT(filter);
for (int i = 0; i < 3; ++i)
{
applyGauss(complexImage[i], filter, side);
}
// Applies IFFT to every channel
applyIFFT(complexImage);
}
private void applyGauss(ComplexImage complexImage, ComplexImage filter, int side)
{
int width = complexImage.Data.GetLength(1);
int height = complexImage.Data.GetLength(0);
for(int i = 0; i < height; ++i)
{
for(int j = 0; j < width; ++j)
{
complexImage.Data[i, j] = AForge.Math.Complex.Multiply(complexImage.Data[i, j], filter.Data[i, j]);
}
}
}
Apply IFFT to every channel.
Convert every channel back to bitmaps (with AForge.NET).
public static System.Drawing.Bitmap[] convertComplexColorChannelsToBitmap(AForge.Imaging.ComplexImage[] channels)
{
System.Drawing.Bitmap[] result = new System.Drawing.Bitmap[channels.Length];
for (int i = 0; i < channels.Length; ++i)
{
result[i] = channels[i].ToBitmap();
}
return result;
}
Merge bitmaps into single bitmap
public static Bitmap mergeColorChannels(Bitmap[] channels)
{
Bitmap result = null;
switch (channels.Length)
{
case 1:
return channels[0];
case 3:
result = new Bitmap(channels[0].Width, channels[0].Height, PixelFormat.Format24bppRgb);
break;
case 4:
result = new Bitmap(channels[0].Width, channels[0].Height, PixelFormat.Format32bppArgb);
break;
default:
throw new NotSupportedException("Bitmap FFTServices.mergeColorChannels(Bitmap[]): Only 1, 3 and 4 channels are supported.");
}
LockBitmap resultLock = new LockBitmap(result);
resultLock.LockBits();
LockBitmap red = new LockBitmap(channels[0]);
LockBitmap green = new LockBitmap(channels[1]);
LockBitmap blue = new LockBitmap(channels[2]);
red.LockBits();
green.LockBits();
blue.LockBits();
for (int y = 0; y < result.Height; y++)
{
for (int x = 0; x < result.Width; x++)
{
resultLock.SetPixel(x, y, Color.FromArgb((int)red.GetPixel(x, y).R, (int)green.GetPixel(x, y).G, (int)blue.GetPixel(x, y).B));
}
}
red.UnlockBits();
green.UnlockBits();
blue.UnlockBits();
resultLock.UnlockBits();
return result;
}
As a result I've got shifted, red-colored blurred version of image: link.
#edit - Updated the question with several changes to the code.
I figured it out with some help at DSP stackexchange... and some cheating but it works. The main problem was kernel generation and applying FFT to it. Also important thing is that AForge.NET divides image pixels by 255 during conversion to ComplexImage and multiplies by 255 during conversion from ComplexImage to Bitmap (thanks Olli Niemitalo # DSP SE).
How I solved this:
I've found how kernel should look like after FFT (see below).
Looked up colors of that image.
Calculated gauss2d for x = -2; y = -2; std = 1.
Calculated the prescaler to receive color value from value calculated in pt. 3 (see wolfram).
Generated kernel with scaled values with perscaler from pt. 4.
However I cant use FFT on generated filter, because generated filter looks like filter after FFT already. It works - the output image is blurred without artifacts so I think that's not too bad.
The images (I cant post more than 2 links, and images are farily big):
Input image
Generated filter (without FFT!)
Parameters for below function:
std = 1.0
size = 8.0
width = height = 512
Result image
The final code:
private ComplexImage makeGaussKernel(double size, double std, int imgWidth, int imgHeight)
{
double scale = 2000.0;
double hsize = size / 2.0;
Bitmap bmp = new Bitmap(imgWidth, imgHeight, PixelFormat.Format8bppIndexed);
LockBitmap lbmp = new LockBitmap(bmp);
lbmp.LockBits();
double y = -hsize;
double yStep = hsize / (lbmp.Height / 2.0);
double xStep = hsize / (lbmp.Width / 2.0);
for (int i = 0; i < lbmp.Height; ++i)
{
double x = -hsize;
for (int j = 0; j < lbmp.Width; ++j)
{
double g = gauss2d(x, y, std) * scale;
g = g < 0.0 ? 0.0 : g;
g = g > 255.0 ? 255.0 : g;
lbmp.SetPixel(j, i, Color.FromArgb((int)g));
x += xStep;
}
y += yStep;
}
lbmp.UnlockBits();
return ComplexImage.FromBitmap(bmp);
}
private double gauss2d(double x, double y, double std)
{
return (1.0 / (2 * Math.PI * std * std)) * Math.Exp(-(((x * x) + (y * y)) / (2 * std * std)));
}
private void applyGaussToImage(ComplexImage complexImage, ComplexImage filter)
{
for (int i = 0; i < complexImage.Height; ++i)
{
for (int j = 0; j < complexImage.Width; ++j)
{
complexImage.Data[i, j] = AForge.Math.Complex.Multiply(complexImage.Data[i, j], filter.Data[i, j]);
}
}
}