I am currently working on a project in which I am required to write software that compares two images made up of the same area and draws a box around the differences. I wrote the program in c# .net in a few hours but soon realized it was INCREDIBLY expensive to run. Here are the steps I implemented it in.
Created a Pixel class that stores the x,y coordinates of each pixel and a PixelRectangle class that stores a list of pixels along with width,height,x and y properties.
Looped through every pixel of each image, comparing the colour of each corresponding pixels. If the colour was different I then created a new pixel object with the x,y coordinates of that pixel and added it to a pixelDifference List.
Next I wrote a method that recursively checks each pixel in the pixelDifference list to create PixelRectangle objects that only contain pixels that are directly next to each other. (Pretty sure this bad boy is causing the majority of the destruction as it gave me a stack overflow error.)
I then worked out the x,y coordinates and dimensions of the rectangle based on the pixels that were stored in the list of the PixelRectangle Object and drew a rectangle over the original image to show where the differences were.
My questions are: Am I going about this the correct way? Would a quad tree hold any value for this project? If you could give me the basic steps on how something like this is normally achieved I would be grateful. Thanks in advance.
Dave.
looks like you want to implement blob detection. my suggestion is not to reinvent the wheel and just use openCVSharp or emgu to do this. google 'blob detection' & opencv
if you want to do it yourself here my 2 cents worth:
first of all, let's clarify what you want to do. really two separate things:
compute the difference between two images (i am assuming they are
the same dimensions)
draw a box around 'areas' that are 'different' as measured by 1. questions here are what is an 'area' and what is considered 'different'.
my suggestion for each step:
(my assumption is both images a grey scale. if not, compute the sum of colours for each pixel to get grey value)
1) cycle through all pixels in both images and subtract them. set a threshold on the absolute difference to determine if their difference is sufficient to represent and actual change in the scene (as opposed to sensor noise etc if the images are from a camera). then store the result in a third image. 0 for no difference. 255 for a difference. if done right this should be REALLY fast. however, in C# you must use pointers to get a decent performance. here an example of how to do this (note: code not tested!!) :
/// <summary>
/// computes difference between two images and stores result in a third image
/// input images must be of same dimension and colour depth
/// </summary>
/// <param name="imageA">first image</param>
/// <param name="imageB">second image</param>
/// <param name="imageDiff">output 0 if same, 255 if different</param>
/// <param name="width">width of images</param>
/// <param name="height">height of images</param>
/// <param name="channels">number of colour channels for the input images</param>
unsafe void ComputeDiffernece(byte[] imageA, byte[] imageB, byte[] imageDiff, int width, int height, int channels, int threshold)
{
int ch = channels;
fixed (byte* piA = imageB, piB = imageB, piD = imageDiff)
{
if (ch > 1) // this a colour image (assuming for RGB ch == 3 and RGBA == 4)
{
for (int r = 0; r < height; r++)
{
byte* pA = piA + r * width * ch;
byte* pB = piB + r * width * ch;
byte* pD = piD + r * width; //this has only one channels!
for (int c = 0; c < width; c++)
{
//assuming three colour channels. if channels is larger ignore extra (as it's likely alpha)
int LA = pA[c * ch] + pA[c * ch + 1] + pA[c * ch + 2];
int LB = pB[c * ch] + pB[c * ch + 1] + pB[c * ch + 2];
if (Math.Abs(LA - LB) > threshold)
{
pD[c] = 255;
}
else
{
pD[c] = 0;
}
}
}
}
else //single grey scale channels
{
for (int r = 0; r < height; r++)
{
byte* pA = piA + r * width;
byte* pB = piB + r * width;
byte* pD = piD + r * width; //this has only one channels!
for (int c = 0; c < width; c++)
{
if (Math.Abs(pA[c] - pB[c]) > threshold)
{
pD[c] = 255;
}
else
{
pD[c] = 0;
}
}
}
}
}
}
2)
not sure what you mean by area here. several solutions depending on what you mean. from simplest to hardest.
a) colour each difference pixel red in your output
b) assuming you only have one area of difference (unlikely) compute the bounding box of all 255 pixels in your output image. this can be done using a simple max / min for both x and y positions on all 255 pixels. single pass through the image and should be very fast.
c) if you have lots of different areas that change - compute the "connected components". that is a collection of pixels that are connected to each other. of course this only works in a binary image (i.e. on or off, or 0 and 255 as in our case). you can implement this in c# and i have done this before. but i won't do this for you here. it's a bit involved. algorithms are out there. again opencv or google connected components.
once you have a list of CC's draw a box around each. done.
You're pretty much going about it the right way. Step 3 shouldn't be causing a StackOverflow exception if it's implemented correctly so I'd take a closer look at that method.
What's most likely happening is that your recursive check of each member of PixelDifference is running infinitely. Make sure you keep track of which Pixels have been checked. Once you check a Pixel it no longer needs to be considered when checking neighbouring Pixels. Before checking any neighbouring pixel make sure it hasn't already been checked itself.
As an alternative to keeping track of which Pixels have been checked you can remove an item from PixelDifference once it has been checked. Of course, this may require a change in the way you implement your algorithm since removing an element from a List while checking it can bring a whole new set of issues.
There's a much simpler way of finding the difference of two images.
So if you have two images
Image<Gray, Byte> A;
Image<Gray, Byte> B;
You can get their differences fast by
A - B
Of course, images don't store negative values so to get differences in cases where pixels in image B are greater than image A
B - A
Combining these together
(A - B) + (B - A)
This is ok, but we can do even better.
This can be evaluated using Fourier transforms.
CvInvoke.cvDFT(A.Convert<Gray, Single>().Ptr, DFTA.Ptr, Emgu.CV.CvEnum.CV_DXT.CV_DXT_FORWARD, -1);
CvInvoke.cvDFT(B.Convert<Gray, Single>().Ptr, DFTB.Ptr, Emgu.CV.CvEnum.CV_DXT.CV_DXT_FORWARD, -1);
CvInvoke.cvDFT((DFTB - DFTA).Convert<Gray, Single>().Ptr, AB.Ptr, Emgu.CV.CvEnum.CV_DXT.CV_DXT_INVERSE, -1);
CvInvoke.cvDFT((DFTA - DFTB).Ptr, BA.Ptr, Emgu.CV.CvEnum.CV_DXT.CV_DXT_INVERSE, -1);
I find that the results from this method are much better.
You can make a binary image out of this, ie: threshold the image so pixels with no change have 0 while pixels that have changes store 255.
Now as far as the second part of the problem goes, I suppose there's a simple crude solution:
Partition the image into rectangular regions. Perhaps there's no need to go as far as using quad trees. Say, an 8x8 grid... (For different results, you can experiment with different grid sizes).
Then use the convex hull function within these regions. These convex hulls can be turned into rectangles by finding the min and max x an y coordinates of their vertices.
Should be fast and simple
Related
This question already has answers here:
How can I measure the similarity between two images? [closed]
(17 answers)
Closed 5 years ago.
I have one Bitmap A and one array of Bitmap, in the array there is a Bitmap that looks the same as Bitmap A. I'm using the code below but it sometimes doesnt work, it iterates the entire array without finding it, it seems there are some minor differences, is there a way to change the function to return true if its 90% similar or pick the most similar image in the array? The array has only 6 images.
for(int i = 0; i < list.Count;i++)
{
if(ImageCompareString(image,list[i])
{
answerIndex = i;
break;
}
}
private static bool ImageCompareString(Bitmap firstImage, Bitmap secondImage)
{
MemoryStream ms = new MemoryStream();
firstImage.Save(ms, System.Drawing.Imaging.ImageFormat.Png);
String firstBitmap = Convert.ToBase64String(ms.ToArray());
ms.Position = 0;
secondImage.Save(ms, System.Drawing.Imaging.ImageFormat.Png);
String secondBitmap = Convert.ToBase64String(ms.ToArray());
if (firstBitmap.Equals(secondBitmap))
{
return true;
}
else
{
return false;
}
}
Of course there is such way... But you have to code it yourself.
First you shoud not compare the base64 data... You'll loose direct pixel value access and increase the size of the data to compare by more then 150% (Originaly 200% but corrected thanks to PeterDuniho's comment) in C# due to UTF16.
Second I assume that all pictures have the same fixed size. Before comparing, reduce the image size to something really small, but keep the width/height aspect. This will speed up the comparsion and also eliminates noise.
Third Iterate both pictures and compare their grayscaled pixel values. I Assume that you have resized the picture to 16x16. Since we're comparing their grayscale-values the value of one pixel is between 0 and 255. So the maximum distance between both pictures will be 16 * 16 * 256 = 65536. If both pictures are black, the distance between the pictures will be zero (100% similarity). If one picture is black and the other is white the distance will be 65535 (0% similarity).
To compare the images iterate the picture-pixels and subtract the grayscale-pixel-value-from-picture-a from the grayscale-pixel-value-of-picture-b at the point x,y and add the absolute difference value to the counter. This counter will be the total distance between both pictures.
Lets assume this counter has a value of 1000 after the comparison loop, you get the percentage-similarity by 1000 / 65535 ~ 1.5% difference (or 98.5% similarity) between both pictures.
pseudo-compare-code
long counter = 0;
long total = image.Width * image.Height * (Color.White - Color.Black);
for(int x = 0; x < image.Width; x++)
{
for(int y = 0; y < image.Height; y++)
{
var p1 = image.GetPixel(x, y);
var p2 = otherImage.GetPixel(x, y);
var g1 = ((p1.R + p1.G + p1.B) / 3);
var g2 = ((p2.R + p2.G + p2.B) / 3);
var distance = Math.Abs(g1 - g2);
counter += distance;
}
}
var similarity = 100 - ((counter / total) * 100);
This is an more or less easy approach, but you have to test this with you scenario/images. Instead of comparing grayscale-values you could also compare rgb-values. Look for distance definitions like the euclidean distance... Start and keep reading :)
EDIT
This is just a really basic approach that should explain how you can start comparing images. It does not take into account that there might be different image formats (jpeg, png, gif), color formats (indexed, 16bit, 24bit, 32bit) or images with different resolutions.
I am trying to recreate the Recolor Picture dialog that Microsoft unfortunately discontinued in the transition from Office 2003 to 2007. This was very useful for replacing colors in a picture (see http://www.indezine.com/products/powerpoint/learn/picturesandvisuals/recolor-pictures-ppt2003.html for full description of dialog).
I am mostly interested in doing this for images in the metafile format (EMF or WMF), which tend to have fewer colors than other picture formats, in my experience. The picture below is an example of an enhanced metafile picture pasted from Excel into PowerPoint that appears to contain just 6 colors:
If I was able to use the legacy Office dialog pictured above, I would see my 6 colors on the left in the "Original" column, and I could easily change the blue font (and border) color to black. The problem here is that if I use GetPixel() to programmatically inventory the colors in the image, I get dozens of colors due to anti-aliasing of the fonts, and it isn't practical to show the user all these recoloring options (which would effectively require the user to manually recreate the proper anti-aliasing effect). The snippet of code below illustrates how I have tried to inventory the colors:
Dim listColors as New List(Of Color)
Dim shp as PowerPoint.Shape = [a metafile picture in PowerPoint]
Dim strTemp as String = Path.Combine(Environ("temp"), "temp_img.emf")
shp.Export(strTemp, PowerPoint.PpShapeFormat.ppShapeFormatEMF, 0, 0)
Using bmp As New Bitmap(strTemp)
For x As Integer = 0 To bmp.Width - 1
For y As Integer = 0 To bmp.Height - 1
listColors.Add(bmp.GetPixel(x, y))
Next
Next
End Using
I see that there is an optional Palette property for metafiles, which I thought could provide an answer, but an exception is thrown when I try to access it, so that was a dead end. I also see that there are headers for metafile images, but I cannot decipher what to do with them from the limited documentation on the Internet, and I am not even sure that that would get me to the right answer (i.e. 6 colors).
To summarize, part 1 of the question is how to inventory (i.e. identify) the 6 "core" colors in the image above, and part 2 is how to replace one of these 6 colors with another. VB.NET solutions are preferred, although I can probably translate C# code if not too complex.
If needed, you can download the EMF version of the image above at https://www.dropbox.com/s/n03ys3dh9pcd0xu/temp_img.emf?dl=0.
EDIT: To be clear, I am not interested in "computing" the six "core" colors in the image above. I believe, perhaps incorrectly, that these six colors are explicit properties of the image, and my first objective is to figure out how to access them. Indeed, if you simply ungroup the metafile picture twice in PowerPoint, you can loop through the resulting shapes to get these six colors. That would address part 1 of the question, although it seems a bit sloppy, works only for metafiles (which may be fine, actually), and I doubt that is how the legacy Recolor Picture dialog worked. To address part 2, I could regroup the metafile picture shapes after swapping colors but, again, that seems sloppy and modifies the picture in ways other than what is intended. So, how can I explicitly retrieve / modify / set these "core" colors in a [metafile] picture?
So I've had a quick stab at implementing one of my ideas to use a dictionary of colours and what not. The code isn't working properly just yet but I figured I show it here so that you can have a quick look into how it works and develop on it from there.
using (Bitmap bitmap = new Bitmap(#"InputPath"))
{
Dictionary<Color, List<Color>> colourDictionary = new Dictionary<Color, List<Color>>();
int nTolerance = 60;
int nBytesPerPixel = Bitmap.GetPixelFormatSize(bitmap.PixelFormat) / 8;
System.Drawing.Imaging.BitmapData bitmapData = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bitmap.PixelFormat);
try
{
int nByteCount = bitmapData.Stride * bitmap.Height;
byte[] _baPixels = new byte[nByteCount];
System.Runtime.InteropServices.Marshal.Copy(bitmapData.Scan0, _baPixels, 0, _baPixels.Length);
int _nStride = bitmapData.Stride;
for (int h = 0; h < bitmap.Height; h++)
{
int nCurrentLine = h * _nStride;
for (int w = 0; w < (bitmap.Width * nBytesPerPixel); w += nBytesPerPixel)
{
int nBlue = _baPixels[nCurrentLine + w];
int nGreen = _baPixels[nCurrentLine + w + 1];
int nRed = _baPixels[nCurrentLine + w + 2];
if (colourDictionary.Keys.Count > 0)
{
Color[] caNearbyColours = colourDictionary.Keys.Select(c => c)
.Where(c => (int)c.B <= (nBlue + nTolerance) && (int)c.B >= (nBlue - nTolerance)
&& (int)c.G <= (nGreen + nTolerance) && (int)c.G >= (nGreen - nTolerance)
&& (int)c.R <= (nRed + nTolerance) && (int)c.R >= (nRed - nTolerance)).ToArray();
if (caNearbyColours.Length > 0)
{
if (!colourDictionary[caNearbyColours.FirstOrDefault()].Any(c => c.R == nRed && c.G == nGreen && c.B == nBlue))
colourDictionary[caNearbyColours.FirstOrDefault()].Add(Color.FromArgb(255, nRed, nGreen, nBlue));
}
else
colourDictionary.Add(Color.FromArgb(255, nRed, nGreen, nBlue), new List<Color>());
}
else
colourDictionary.Add(Color.FromArgb(255, nRed, nGreen, nBlue), new List<Color>());
}
}
}
finally
{
bitmap.UnlockBits(bitmapData);
}
using (Bitmap colourBitmap = new Bitmap(bitmap.Width, bitmap.Height, bitmap.PixelFormat))
{
using (Graphics g = Graphics.FromImage(colourBitmap))
{
for (int h = 0; h < colourBitmap.Height; h++)
{
for (int w = 0; w < colourBitmap.Width; w++)
{
Color colour = bitmap.GetPixel(w, h);
if (!colourDictionary.ContainsKey(colour))
{
Color keyColour = colourDictionary.Keys.FirstOrDefault(k => colourDictionary[k].Any(v => v == colour));
colourBitmap.SetPixel(w, h, keyColour);
}
else
colourBitmap.SetPixel(w, h, colour);
}
}
}
colourBitmap.Save(#"OutputPath", System.Drawing.Imaging.ImageFormat.Png);
}
}
Notice how the top section uses Lockbits for better performance. This can easily be transferred onto the bottom section. Also note that the lock bits code is set to work for images with a bytes per pixel of 3 or 4.
Now onto how the code is attempting to work:
It starts off by looping over the initial image and finding colours. If the colour is found already it skips it, however if the colour is not in the dictionary it will add it and also if the colour has a similar colour in the dictionaries keys (will need changing to look at values too) it will add it to it's values.
This then loops through the pixels in the output image setting them according to the keys in the dictionary, thus creating a 'blocked' image.
Now as I said it isn't working just yet, so here are some improvements needed to be made:
As said above checking the colour to those in the values
Adding lock bits to the bottom code for better performance
Tweaking the nTolerance value for better results
Keep track of counts of colours and at the end of looping set the key to that with the largest count
And of course anything else that I have not thought of
I've been browsing the Web for an algorithm which analyzes an image and returns the most different and used colors -- but with no luck.
For instance;
If I'd have a bitmap with 25% red (255,0,0) in RGB, 50% fully blue (0,0,255) and 25% pink (255,0,255)
I would want the algorithm to return these three (or more, depending on avalible colors) colors sorted in the usage, so:
1. Blue
2. Red / Pink
3. Red / Pink
Anyone know some way I could start this? Maybe some articles to read etc. I've never used images in C# like this.
If I understood the question correctly, this may help you (of course you have to read the data according to your needs then):
/// <summary>
/// Gets the bitmap image color statistics
/// </summary>
/// <param name="bit">The bitmap image you want to analyze</param>
/// <returns></returns>
public static List<KeyValuePair<Color, int>> GetStatistics(Bitmap bit)
{
Dictionary<Color, int> countDictionary = new Dictionary<Color, int>();
for (int wid = 0; wid < bit.Width; wid++)
{//for every column in the image
for (int he = 0; he < bit.Height; he++)
{//for every row in the image
//Get the color of the pixel reached (i.e. at the current column and row)
Color currentColor = bit.GetPixel(wid, he);
//If a record already exists for this color, set the count, otherwise just set it as 0
int currentCount = (countDictionary.ContainsKey(currentColor) ? countDictionary[currentColor] : 0);
if (currentCount == 0)
{//If this color doesnt already exists in the dictionary, add it
countDictionary.Add(currentColor, 1);
}
else
{//If it exists, increment the value and update it
countDictionary[currentColor] = currentCount + 1;
}
}
}
//order the list from most used to least used before returning
List<KeyValuePair<Color, int>> l = countDictionary.OrderByDescending(o => o.Value).ToList();
return l;
}
}
most used colors
With a little google effort you would find Histogram. If you want to use shades as separate color then you have 256^3 colors. So either use some dynamic list instead or ignore few least significant bits to lower the number a bit. You can also change the dynamic range by normalization of colors.
black is black
and for everything else change vector size to Max for example
Max = 2^5-1 = 31
normalized color = color * Max / |color|
Now the algorithm:
create a counter table cnt for all combinations of colors
for Max = 31 the size would be 2^15 = 32768. Set the whole table to zero.
int cnt[32768];
for (int i=0;i<32768;i++) cnt[i]=0;
go through entire image and for each pixel
normalize its color
convert it to address (for example adr = (R) | (G<<5) | (B<<10))
increment its counter cnt[adr]++;
After this you have histogram in cnt[]. so now index sort it by value of cnt and you have obtained the colors sorted by their usage
most different color
How would you define it? I would use data from histogram and do search for max distance between 2 colors in it (after normalization)
d = |color1 - color2|
Do not need to sqrt it ... if you use d^2 you will obtain the same results. Ignore all entries where cnt[adr]==0 (that is unused color). This is still O(n^2) ... more like ~T(n*n/2) ... in runtime terms but n is not the number of pixels in image. instead it is just number of different used colors in image which is far less ... Also after index sort of histogram and removing/ignoring all the cnt[adr]==0 entries even lover.
I have a function to check if an image is just one color.
bool r = true;
Color checkColor = image.GetPixel(0, 0);
for (int x = 0; x < image.Width; x++)
{
for (int y = 0; y < image.Height; y++)
{
if (image.GetPixel(x, y) != checkColor) { r = false; }
}
}
// image color
clrOut = checkColor;
return r;
But this algorithm is slow for big images.
Does anyone knows a way to do this using Pixel Shaders and GPU?
You don't need pixel shaders and a GPU to speed this up. Use LockBits. Bob Powell has a good tutorial on doing exactly what you want.
Also looking at your code, try reversing the for loops it gives better memory access
for( y...
...
for ( x...
The next step is to unroll some of the pixels access. Try fetching 4 or 8 pixels in the inter loop
for( y...
...
for ( x = 0; x < image.Width ; x +=4 )
pixel0 = image.getpixel(x,Y)
pixel1 = image.getpixel(x +1,Y)
pixel2 = image.getpixel(x +2,Y)
pixel3 = image.getpixel(x +3,Y)
if ( pixel0 ....
As stated earlier using Bitmap Unlock allows you access pixels via pointers, which is the fastest you can get on a CPU. You can apply loop ordering and pixel unrolling to that technique too.
If this isn't fast enough then there is choice between; C# multi-threading or GPU with OpenCL and its C# binding.
This code is slow, because you use GetPixel. You can make it much faster by using direct pointer access. Only if that's not enough, I'd look into pixel shaders.
I've written some helper libraries: https://github.com/CodesInChaos/ChaosUtil/tree/master/Chaos.Image
In particular the Pixels and the RawColor types should be useful.
If you're only looking for an image with large areas of variation versus one with the same color, you could shrink it down to a 1x1 pixel using Bilinear filtering and read pixel 0,0.
If the the pixel is VERY different from what you expect (RGB distance versus a tolerance), you can be sure that there was some variation in the original image.
Of course, this depends on what you really want to do with this info so YMMV.
i am coding a software at visual studio 2010 by using C#. what does this software do is finding same squares at the image after a square selected. every square is composed by 16x16 pixel. my current algorithm starts from first pixel and scan the entire image pixel by pixel comparing to determine same pixel squares with the selected one. this takes really big time. Can you suggest me any better way ?
also every square is ordered. so they start like 0 - 16 - 32 - 48
a square can not start from 5 or 65 etc
thank you
You could cache a checksum of each image-region. Then you would only have to check ones that match the checksum for equality.
Let's assume each image is 16x16 rgb elements. You could do this (and yes, it will have integer overflow.)
All of this is in pseudo code - you're expected to be able to translate this to C#.
Add an int to the field image class, or create an image wrapper with an int as the 'checksum'
int checksum = 0
for each pixel in image {
checksum += pixel.red + pixel.blue + pixel.green
// you could do anything you wanted here, like
// checksum *= 17 + pixel.red
// checksum *= 17 + pixel.blue
// checksum *= 17 + pixel.green
// just make it "unique enough", like a hashcode
}
image.checksum = checksum
Now when you go to search you can go like this:
/**
* equals method before:
*/
boolean equals(image a, image b) {
for x = 0..15 do /* all 16 pixels in X */
for y = 0..15 do /* all 16 pixels in Y */
if a.getPixel(x,y) != b.getPixel(x,y) return false;
return true;
}
/**
* equals method after:
*.
boolean equals(image a, image b) {
/* this check lets you skip the loop in most cases */
/* still have to verify that the image is equal pixel for pixel though */
if a.checksum != b.checksum return false;
for x = 0..15 do /* all 16 pixels in X */
for y = 0..15 do /* all 16 pixels in Y */
if a.getPixel(x,y) != b.getPixel(x,y) return false;
return true;
}
One algorithm I know of to compare how similar two images are is the root mean square algorithm. I've used this in several programs and it's always been pretty fast. This simply sums up the differences, does some math, and the difference will be 'how close' the two images are to each other.
But, if the access is slow for comparison of each pixel, then access will still be slow (albeit slightly faster) adding them all up (or checksumming it).
Another option would be to add a short-circuit. The second any pixel at all doesn't match, you can claim the whole picture doesn't match.
I would be curious why this is going slow, though. The image would have to be incredibly large for it to not count. Are you using Bitmap.GetPixel()?