Develop Reference

CrossCorrelation of images

I used the Accord Framework to implement the cross correlation between two images. My goal is to find by how much (how many pixels, and in which direction) the second image is shifted compared to the first one.
The basic formula I used is the following :
corr(a, b) = ifft(fft(a_and_zeros) * conj(fft(b_and_zeros)))
I'll put the whole code at the end of my message, everything happens on a Click event. My initial images were stored in 1024*768 bitmaps. So here is the steps I have taken :
I cropped the 2 images into the 4 zones that were interesting for me (ExpInit, ExpFinal, RefInit, RefFinal), that I want to correlate two by two (ExpInit with ExpFinal and RefInit with RefFinal). Those cropped images have dimensions of 1024*131.
I put those cropped images in the center of new bitmaps with 2^n dimensions (2048*512).
Applied a Grayscaling filter to get 8bppIndexed PixelFormat.
Converted each image to ComplexImageformat and applied forward FFT on the 4 images.
Complex-conjugating every elements in the RefFinal and ExpFinal fourier-transformed ComplexImage.
Execute element-wise multiplication between the ComplexImageobjects I want to cross-correlate (ExpInit with ExpFinal, RefInit wit RefFinal).
Apply the backward FFT to the product of the element-wise multiplication. Tadaaa, my cross-correlation is done, and I have two Complex[,] objects with the dimensions of my images (2048*512 pixels)
Now I want to answer my initial question : by how much (how many pixels, and in which direction) is the ExpFinal (respectively RefFinal) image shifted compared to the ExpInit (respectively ExpFinal). Here I am left puzzled.
I have the intutition I should be drawing a 3D graph with my Complex[,] object, where x and y are the index in the array, and z the value at the index, and search for the max value, but how do I do that with complex numbers ? Do I use only the Re part ? Only the Im part ? The amplitude ? Or am I completely mistaken ?
Bonus question : what is a good library for drawing such graphs ?
Here is the whole code for the described cross-correlation :
private void crosscorrButton_Click(object sender, EventArgs e)
{
// Cropping all 4 sections (RefInit, ExpInit, RefFinal, ExpFinal) and placing them in the center of new Bitmaps with 2^n dimensions
Rectangle rExp = new Rectangle(1, 157, 1024, 131);
Bitmap ExpInitCrop = new Bitmap(rExp.Width, rExp.Height);
Graphics g = Graphics.FromImage(ExpInitCrop);
g.DrawImage(BMInit, -rExp.X, -rExp.Y);
Bitmap ExpInitLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(ExpInitLarge))
{
largeGraphics.DrawImage(ExpInitCrop, 513, 190);
}
Rectangle rRef = new Rectangle(1, 484, 1024, 131);
Bitmap RefInitCrop = new Bitmap(rRef.Width, rRef.Height);
Graphics h = Graphics.FromImage(RefInitCrop);
h.DrawImage(BMInit, -rRef.X, -rRef.Y);
Bitmap RefInitLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(RefInitLarge))
{
largeGraphics.DrawImage(RefInitCrop, 513, 190);
}
Bitmap ExpFinalCrop = new Bitmap(rExp.Width, rExp.Height);
Graphics i = Graphics.FromImage(ExpFinalCrop);
i.DrawImage(BMFinal, -rExp.X, -rExp.Y);
Bitmap ExpFinalLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(ExpFinalLarge))
{
largeGraphics.DrawImage(ExpFinalCrop, 513, 190);
}
Bitmap RefFinalCrop = new Bitmap(rRef.Width, rRef.Height);
Graphics j = Graphics.FromImage(RefFinalCrop);
j.DrawImage(BMFinal, -rRef.X, -rRef.Y);
Bitmap RefFinalLarge = new Bitmap(2048, 512);
using (Graphics largeGraphics = Graphics.FromImage(RefFinalLarge))
{
largeGraphics.DrawImage(RefFinalCrop, 513, 190);
}
// Grayscalling the 4 sections to get 8bppIndexed PixelFormat
Accord.Imaging.Filters.Grayscale filterGS = new Accord.Imaging.Filters.Grayscale(0.2125, 0.7154, 0.0721);
Bitmap RefFinalLargeGS = filterGS.Apply(RefFinalLarge);
Bitmap ExpFinalLargeGS = filterGS.Apply(ExpFinalLarge);
Bitmap RefInitLargeGS = filterGS.Apply(RefInitLarge);
Bitmap ExpInitLargeGS = filterGS.Apply(ExpInitLarge);
// FFT on the 4 sections
Accord.Imaging.ComplexImage ExpInitComplex = Accord.Imaging.ComplexImage.FromBitmap(ExpInitLargeGS);
ExpInitComplex.ForwardFourierTransform();
Accord.Imaging.ComplexImage RefInitComplex = Accord.Imaging.ComplexImage.FromBitmap(RefInitLargeGS);
RefInitComplex.ForwardFourierTransform();
Accord.Imaging.ComplexImage ExpFinalComplex = Accord.Imaging.ComplexImage.FromBitmap(ExpFinalLargeGS);
ExpFinalComplex.ForwardFourierTransform();
Accord.Imaging.ComplexImage RefFinalComplex = Accord.Imaging.ComplexImage.FromBitmap(RefFinalLargeGS);
RefFinalComplex.ForwardFourierTransform();
//Conjugating the ExpFinal and RefFinal section
Complex[,] CompConjExpFinal = new Complex[ExpFinalComplex.Height, ExpFinalComplex.Width];
Complex[,] CompConjRefFinal = new Complex[RefFinalComplex.Height, RefFinalComplex.Width];
for (int l = 0; l < ExpFinalComplex.Height; l++)
{
for (int m = 0; m < ExpFinalComplex.Width; m++)
{
CompConjExpFinal[l, m] = System.Numerics.Complex.Conjugate(ExpFinalComplex.Data[l, m]);
ExpFinalComplex.Data[l, m] = CompConjExpFinal[l, m];
}
}
for (int l = 0; l < RefFinalComplex.Height; l++)
{
for (int m = 0; m < RefFinalComplex.Width; m++)
{
CompConjRefFinal[l, m] = System.Numerics.Complex.Conjugate(RefFinalComplex.Data[l, m]);
RefFinalComplex.Data[l, m] = CompConjRefFinal[l, m];
}
}
//Element-wise multiplication of the complex arrays two by two
Complex[,] ExpMultipliedMatrix = new Complex[ExpFinalComplex.Height, ExpFinalComplex.Width];
Complex[,] RefMultipliedMatrix = new Complex[RefFinalComplex.Height, RefFinalComplex.Width];
for (int l = 0; l < ExpFinalComplex.Height; l++)
{
for (int m = 0; m < ExpFinalComplex.Width; m++)
{
ExpMultipliedMatrix[l, m] = System.Numerics.Complex.Multiply(ExpInitComplex.Data[l, m], ExpFinalComplex.Data[l, m]);
RefMultipliedMatrix[l, m] = System.Numerics.Complex.Multiply(RefInitComplex.Data[l, m], RefFinalComplex.Data[l, m]);
}
}
//InverseFFT
Complex[,] CrossCorrExpMatrix = new Complex[ExpFinalComplex.Height, ExpFinalComplex.Width];
Complex[,] CrossCorrRefMatrix = new Complex[RefFinalComplex.Height, RefFinalComplex.Width];
Accord.Math.FourierTransform.FFT2(ExpMultipliedMatrix, FourierTransform.Direction.Backward);
Accord.Math.FourierTransform.FFT2(RefMultipliedMatrix, FourierTransform.Direction.Backward);
CrossCorrExpMatrix = ExpMultipliedMatrix;
CrossCorrRefMatrix = RefMultipliedMatrix;
}
Thanks a lot !

The imaginary part of the result should be 0 (or within numerical error). To find the shift you should be looking at the location of the peak of the correlation's amplitude (but unless you've got one is the negative image of the other, that's likely to correspond to the peak of the correlation's real part). The main thing to be careful about: since you centered both images, an extra shift (of half the image size) will be introduced.
As for viewing the graph, you could fairly easily map the result to a grayscale image and view it with your favorite image viewer.

Implementation of Bit-Plane Decomposition c#

Tried to implement a bpcs-steganography, and one of the first problem i faced was bit-plane decomposition.
I've created a method (GetBitPlaneRed) (only for Red yet, but it seems the same for other colors) which creates a red-and-white bitmap based on the original bitmap and the index of bit plane (from 1 to 8).
private static int GetBit(byte b, int bitIndex)
{
return (b >> bitIndex) & 0x01;
}
private static Bitmap GetBitPlaneRed(Bitmap bitmap, int bitPlaneIndex)
{
Bitmap newBitmap = new Bitmap(bitmap.Width, bitmap.Height);
for (int i = 0; i < bitmap.Width; i++)
{
for (int j = 0; j < bitmap.Height; j++)
{
Color currColor = bitmap.GetPixel(i, j);
int bit = GetBit(currColor.R, bitPlaneIndex);
Color newColor = Color.FromArgb(255, 255 * bit, 255 * bit);
newBitmap.SetPixel(i, j, newColor);
}
}
return newBitmap;
}
Seems, it works allright for the MSB (most significant bit), but for other bit planes it's not so correct. Here are some result pictures that i've got to comparison with the right ones.
[EDIT] The "right results" are from a scientific article about BPCS steganography written by Eiji Kawaguchi, so i trust that source. Also, it seems that the mistake is in the way i save my bit-plane images, so i've added some peace of code here where i save my bit-plane images.
Added an original image as well.
private static void SaveBitPlanes()
{
string filePath = "monalisa.jpg";
string ext = System.IO.Path.GetExtension(filePath);
Bitmap bitmap = new Bitmap(filePath);
ImageFormat imageFormat = bitmap.RawFormat;
for (int i = 0; i < 8; i++)
{
Bitmap newBitmap = GetBitPlaneRed(bitmap, i);
newBitmap.Save("bitPlaneRed" + i + ext, imageFormat);
}
}
Original image:
My result of MSB plane:
My result of bit plane #3:
My result of bit plane #7:
Right results:
I would appreciate any help or advice.

There are all important stuff I figured out during existance of this question.
All code I wrote in main post is correct.
Difference in images is not connected with jpeg compression or any
other compression. It works for any file format.
The reason why my results were not similar to those results in an
article is that I used a low-quality image as a source. After I tried
some other high quality images, results look nice and pretty
detailed.
Thank everybody for helping me in figuring out all these points!

Windows.Forms - Change ImageList Bitmaps pixels for HighContrast

Need help with altering the Bitmaps in an ImageList to switch the black pixels to yellow, they are being used as images for a UICommandBar (screenshot and attempt below)
The code gets executed, with GetPixel condition and then SetPixel, but the images are not changed. Maybe related to ImageListStreamer or not the right time to swap those pixels.
rtfeditor.cs
this.imageList1.ImageStream = ( (System.Windows.Forms.ImageListStreamer)(resources.GetObject( "imageList1.ImageStream" ) ) );
this.imageList1 = ColorSpace.ProcessHighContrast(this.imageList1);
Utils\ColorSpace.cs
public static System.Windows.Forms.ImageList ProcessHighContrast(System.Windows.Forms.ImageList imageList)
{
if (System.Windows.Forms.SystemInformation.HighContrast)
{
foreach (System.Drawing.Bitmap imageListImage in imageList.Images)
{
for (int i = 0; i < imageListImage.Width; i++)
{
for (int j = 0; j < imageListImage.Height; j++)
{
Color color = imageListImage.GetPixel(i, j);
if (System.Drawing.Color.FromArgb(255, 0, 0, 0).Equals(color))
imageListImage.SetPixel(i, j, System.Drawing.SystemColors.WindowText);
}
}
}
}
return imageList;
}

Solved
Hans Passant's comment was critical for resolving this (see quoted comment below) + changing the order of the call of ProcessHighContrast (now first) with the assignment of the ImageList to the Janus.Windows.UI.CommandBars.UICommandBar (now second). New method and calling code below.
ImageList.Images returns a copy of each image. Altering the copy therefore does not modify the ImageList content at all. Create a new ImageList instead. Do consider that Darth Vader color schemes tend to be appreciated only by programmers, regular users expect the high contrast version to have a white background. That's going to be a lot less painful. – Hans Passant
rtfeditor.cs
this.imageList1.ImageStream = ((System.Windows.Forms.ImageListStreamer)(resources.GetObject("imageList1.ImageStream")));
this.imageList1 = ColorSpace.ProcessHighContrast(this.imageList1);
this.uiCommandManager1.ImageList = this.imageList1;
Utils\ColorSpace.cs
public static System.Windows.Forms.ImageList ProcessHighContrast(System.Windows.Forms.ImageList imageList)
{
if (System.Windows.Forms.SystemInformation.HighContrast)
{
System.Windows.Forms.ImageList imageListNew = new ImageList();
foreach (System.Drawing.Bitmap imageListImage in imageList.Images)
{
for (int i = 0; i < imageListImage.Width; i++)
{
for (int j = 0; j < imageListImage.Height; j++)
{
Color color = imageListImage.GetPixel(i, j);
if (System.Drawing.Color.FromArgb(255, 0, 0, 0).Equals(color))
imageListImage.SetPixel(i, j, System.Drawing.SystemColors.WindowText);
}
}
imageListNew.Images.Add(imageListImage);
}
return imageListNew;
}
return imageList;
}

Attempting to draw rectangles multithreaded

I am attempting to draw about 3600 points on a form, it is pretty slow using one thread so I decided I want to use 4 threads for it.
In my code I divide the 3600 points to the 4 threads and they are supposed to draw it. however for some reason an ArgumentOutOfRangeException is being thrown.
I tried to debug my code but I couldn't find the mistake.
here is the code :
(Ignore the class _3DPoint, it is just a point that has x,y,z values. when I draw them I only use the x,y values.)
code for drawing the points :
public Graphics g; //g = this.CreateGraphics() in form1.Load()
public void drawrectangle(_3DPoint)
float xCord = float.Parse(p.x.ToString());
float yCord = float.Parse(p.y.ToString());
Brush b = new SolidBrush(Color.White);
xCord = lsize * xCord + center.X;
yCord = lsize * yCord + 10 + center.Y;
g.FillRectangle(b, xCord, yCord, 2, 2);
}
lsize, center are just variables for aligning the points as I want them.
All of the multithread action code:
public List<_3DPoint[]> multiThreadsdata = new List<_3DPoint[]>();
public void handlemultithread(_3DPoint[] P)
{
g.Clear(Color.Black);
for (int i = 0; i < multiThreads.Length; i++)
{
multiThreadsdata.Add(new _3DPoint[P.Length / multiThreads.Length]);
}
for (int i = 0; i < multiThreads.Length; i++)
{
for (int j = (P.Length / multiThreads.Length) * (i); j < (P.Length / multiThreads.Length) * (i + 1); j++)
{
multiThreadsdata[i][j - ((P.Length / multiThreads.Length) * i)] = new _3DPoint(P[j]);
}
}
for (int i = 0; i < multiThreads.Length; i++)
{
multiThreads[i] = new Thread(() => drawPoints(multiThreadsdata[i]));
multiThreads[i].Start();
}
}
delegate void SetCallBackPoint(_3DPoint location);
public void drawPoints(_3DPoint[] locations)
{
for (int i = 0; i < locations.Length; i++)
{
if (this.InvokeRequired)
{
SetCallBackPoint e = new SetCallBackPoint(drawrectangle);
this.Invoke(e, new object[] { locations[i] });
}
else
{
drawrectangle(locations[i]);
}
}
}
P is a _3DPoint array that contains all the 3600 points.
mutliThreads is a Thread[] containing 4 threads.
I get the exception in handlemultithread method. in the third line of this for loop :
for (int i = 0; i < multiThreads.Length; i++)
{
multiThreads[i] = new Thread(() => drawPoints(multiThreadsdata[i])); // <- here.
multiThreads[i].Start();
}
I don't know what is the problem, my guess is that there is some problem with the multithreading because I'm just a beginner with multithreading.
Thanks a bunch.

It is entirely possible to Draw 3600 rectangles quickly on a form when you apply the suggestions in the comments.
If that doesn't give you enough time you can consider creating Images on a single background thread, store them in some sort of buffer until they are needed to e painted on the Graphics object of the Paint event of the form. That is only feasible if you can know upfront what needs to be painted on the next frame.
This example uses a simple Background worker to fill an ConcurrentQueue with images. The comments in the code explain what is going on.
public partial class Form1 : Form
{
static ConcurrentQueue<Image> buffer = new ConcurrentQueue<Image>();
static Random r = new Random();
public Form1()
{
InitializeComponent();
backgroundWorker1.RunWorkerAsync();
// this is already a great performance win ...
DoubleBuffered = true;
}
private void Form1_Paint(object sender, PaintEventArgs e)
{
Image img =null;
// get from buffer ..
if (!buffer.TryDequeue(out img))
{
// nothing available
// direct random
for (var x = 0; x < e.ClipRectangle.Width; x++)
{
for (var y = 0; y < e.ClipRectangle.Height; y++)
{
using (var pen = new Pen(new SolidBrush(Color.FromArgb(r.Next(255), r.Next(255), r.Next(255)))))
{
e.Graphics.DrawRectangle(pen, x, y, 1, 1);
}
}
}
}
else
{
// otherwise Draw the prepared image
e.Graphics.DrawImage(img,0,0);
Trace.WriteLine(buffer.Count);
img.Dispose();
}
}
private void button1_Click(object sender, EventArgs e)
{
// force a repaint of the Form
Invalidate();
}
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
// as long as the form is not disposed
while (!IsDisposed)
{
// we keep 60 images in memory
if (buffer.Count < 60)
{
// bitmap
var bmp = new Bitmap(this.Width, this.Height);
var img = Graphics.FromImage(bmp);
// draw
for (int i = 0; i < 3600; i++)
{
using (var pen = new Pen(new SolidBrush(Color.FromArgb(r.Next(255), r.Next(255), r.Next(255)))))
{
img.DrawRectangle(pen, r.Next(Width),r.Next(Height), r.Next(Width), r.Next(Height));
}
}
// store the drawing in the buffer
buffer.Enqueue(bmp);
}
else
{
// simple and naive way to give other threads a bit of room
Thread.Sleep(0);
}
}
}
}
Keep in mind that when you have a CPU heavy process adding more threads will not by magic make your methods run quicker. You might even make it worse: more threads compete for time on the CPU.

Need something to draw many images quickly in WPF

I'm working on Generator of cellular automata in .NET 3.5. I decided to use WPF because I wanted to learn something new in WPF.
I draw OneDimensional automata like Rule 30 and TwoDimensional like Life.
I need something to draw many images quick.
For example, dimensions of mesh is 64 x 64 and size of cell is 12px. So I draw 64*64 = 4096 images in one step. And interval between one step is about 100 ms.
I rewrite my application to the WinForms and there is everything ok. But in WPF is slow and I don't know why.
My example of drawing in WPF:
I have a class derived from Image. In this class I draw a bitmap and I use it in Source property.
public void DrawAll()
{
DrawingGroup dg = new DrawingGroup();
for (byte i = 0; i < this.DimensionX; ++i)
{
for (byte j = 0; j < this.DimensionY; ++j)
{
Piece p = this.Mesh[i][j];
Rect rect = new Rect(j * this.CellSize + (j + 1), i * this.CellSize + (i + 1),
this.CellSize, this.CellSize);
ImageDrawing id = new ImageDrawing();
id.Rect = rect;
if (p == null)
id.ImageSource = this._undefinedPiece.Image.Source;
else
id.ImageSource = p.Image.Source;
dg.Children.Add(id);
}
}
DrawingImage di = new DrawingImage(dg);
this.Source = di;
}
Second example of drawing in WPF:
I have derived class from Canvas and override OnRender function. Based on this article: http://sachabarber.net/?p=675
protected override void OnRender(DrawingContext dc)
{
base.OnRender(dc);
for (byte i = 0; i < this.DimensionX; ++i)
{
for (byte j = 0; j < this.DimensionY; ++j)
{
Rect rect = new Rect(j * this.CellSize + (j + 1), i * this.CellSize + (i + 1),
this.CellSize, this.CellSize);
BitmapImage bi;
int counter = i + j + DateTime.Now.Millisecond;
if (counter % 3 == 0)
bi = this.bundefined;
else if (counter % 3 == 1)
bi = this.bwhite;
else
bi = this.bred;
dc.DrawImage(bi, rect);
++counter;
}
}
}
Thanks for all replies

As written in this article, the fastest way to draw 2D in wpf is the StreamGeomerty.
I'm sure you're able to accomodate your pattern generation logic to the needs of this class.

WPF real power is in rendering vector. Is there a way that instead of using Image, you can create rectangle objects of a predetermined size assign color and then assign location? Bitmap is pretty resource intensive to work with vs vector. Even though WPF pushes the rendering off to the video card, it is still a lot of processing.

You can create an 'Icon Font' which would contain your images instead of typical characters. Displaying images through the font has no difference from using it for regular text.
<TextBlock Text="&#xE726" FontFamily="Segoe MDL2 Assets" Foreground="Red"/>
Advantages:
Glyphs scale losslessly
Different colors can be set
Better performance then Shapes / Paths