Develop Reference

How to compare 2 images and compare them

I am making a video recorder, The app works by taking a lot of screenshots and putting them together into one video. Also, I am trying to make something like screen motion detection. I need the app to take screenshots only when a difference in the screen is detected. I was thinking about how to do that, and I believe I need to make it still take screenshots while comparing them to the previous one. Is there a way to do that?
The code:
//Record video:
public void RecordVideo()
{
//Keep track of time:
watch.Start();
using (Bitmap bitmap = new Bitmap(bounds.Width, bounds.Height))
{
using (Graphics g = Graphics.FromImage(bitmap))
{
//Add screen to bitmap:
g.CopyFromScreen(new Point(bounds.Left, bounds.Top), Point.Empty, bounds.Size);
}
//Save screenshot:
string name = tempPath + "//screenshot-" + fileCount + ".png";
bitmap.Save(name, ImageFormat.Png);
inputImageSequence.Add(name);
fileCount++;
//Dispose of bitmap:
bitmap.Dispose();
}
}

I have something that may be useful for you. The idea is save only the differences between the images and, with that, recreate later all images from starting image and saved changes.
To do this, you only need make a XOR operation in the image bytes. This method allow you get the difference (the array parameter) between two images:
protected void ApplyXor(Bitmap img1, Bitmap img2, byte[] array)
{
const ImageLockMode rw = ImageLockMode.ReadWrite;
const PixelFormat argb = PixelFormat.Format32bppArgb;
var locked1 = img1.LockBits(new Rectangle(0, 0, img1.Width, img1.Height), rw, argb);
var locked2 = img2.LockBits(new Rectangle(0, 0, img2.Width, img2.Height), rw, argb);
try
{
ApplyXor(locked2, locked1, array);
}
finally
{
img1.UnlockBits(locked1);
img2.UnlockBits(locked2);
}
}
With the previous img1 bitmap and the array returned, you can get the img2 with this method:
protected void ApplyXor(Bitmap img1, byte[] array, Bitmap img2)
{
const ImageLockMode rw = ImageLockMode.ReadWrite;
const PixelFormat argb = PixelFormat.Format32bppArgb;
var locked1 = img1.LockBits(new Rectangle(0, 0, img1.Width, img1.Height), rw, argb);
var locked2 = img2.LockBits(new Rectangle(0, 0, img2.Width, img2.Height), rw, argb);
try
{
ApplyXor(locked1, array, locked2);
}
finally
{
img1.UnlockBits(locked1);
img2.UnlockBits(locked2);
}
}
And here the other required methods:
private unsafe void ApplyXor(BitmapData img1, BitmapData img2, byte[] array)
{
byte* prev0 = (byte*)img1.Scan0.ToPointer();
byte* cur0 = (byte*)img2.Scan0.ToPointer();
int height = img1.Height;
int width = img1.Width;
int halfwidth = width / 2;
fixed (byte* target = array)
{
ulong* dst = (ulong*)target;
for (int y = 0; y < height; ++y)
{
ulong* prevRow = (ulong*)(prev0 + img1.Stride * y);
ulong* curRow = (ulong*)(cur0 + img2.Stride * y);
for (int x = 0; x < halfwidth; ++x)
{
if (curRow[x] != prevRow[x])
{
int a = 0;
}
*(dst++) = curRow[x] ^ prevRow[x];
}
}
}
}
private unsafe void ApplyXor(BitmapData img1, byte[] array, BitmapData img2)
{
byte* prev0 = (byte*)img1.Scan0.ToPointer();
byte* cur0 = (byte*)img2.Scan0.ToPointer();
int height = img1.Height;
int width = img1.Width;
int halfwidth = width / 2;
fixed (byte* target = array)
{
ulong* dst = (ulong*)target;
for (int y = 0; y < height; ++y)
{
ulong* prevRow = (ulong*)(prev0 + img1.Stride * y);
ulong* curRow = (ulong*)(cur0 + img2.Stride * y);
for (int x = 0; x < halfwidth; ++x)
{
curRow[x] = *(dst++) ^ prevRow[x];
}
}
}
}
NOTE: You must configure your project to allow unsafe.
With previous methods, you can do:
Save a img1 bitmap
Get img2 bitmap, do XOR and get the array (array2, for example)
With img3, get the XOR with img2 (array3, for example). Now, img2 isn't needed
With img4, get the XOR with img3 (array4). Now, img3 isn't needed
...
You have img1 and array2, array3, array4... and you can recreate all images:
Make XOR between img1 and array2 to get img2
Make XOR between img2 and array3 to get img3
...
If you need send video over TCP, you can send the images sending one image and the XOR arrays (the differences). Or better yet, compress the XOR arrays using K4os.Compression.LZ4.

Why AccessViolationException occurs when accessing pixels in WriteableBitmap Image?

I have a video stream from a camera to an Image in a WPF. I am trying to access the WritableBitMap Image pixel by pixel before displaying it. As a test I am trying to set the whole image to white or black. In both cases however, I get the AccessViolationException error.
I checked other posts and it seems that this error is very wide and not specific to my case. I can't seem to know why I am not getting this working.
So what is the best way to play with the pixels in my case? or why this is not working? Any help is appreciated
private async void UpdateMasterCameraPreview(IdsFrame frame)
{
if (masterImage != null)
frame.Image.CopyTo(masterImage);
else
masterImage = frame.Image.ToWriteableBitmap();
//BitmapImage temp = ConvertWriteableBitmapToBitmapImage(masterImage);
WriteableBitmap temp = masterImage;
// Here I get the exception, on every pixel access
for (int y = 0; y < temp.PixelHeight; y++)
for (int x = 0; x < temp.PixelWidth; x++)
temp.SetPixel(x, y, 255);
masterImage = temp;
masterImage.Lock();
masterImage.AddDirtyRect(new Int32Rect(0, 0, masterImage.PixelWidth, masterImage.PixelHeight));
masterImage.Unlock();
if (OnMasterFrameCaptured != null)
OnMasterFrameCaptured(this, new CameraFrameCapturedArgs(CameraType.Master, masterImage));
}

You have swapped X and Y, i represents height, j represents width, then you shouldcall SetPixel like:
temp.SetPixel(j, i, 255);
On cases like this is better to use meaningful names for variables, like X and Y.

I ended up using the answer of this post. I now can edit raw pixel data of any WriteableBitmap image before sending it to image control in WPF. Below is what I exactly used but here I just convert every frame to some transparency under a condition:
public void ConvertImage(ref WriteableBitmap Wbmp)
{
int width = Wbmp.PixelWidth;
int height = Wbmp.PixelHeight;
int stride = Wbmp.BackBufferStride;
int bytesPerPixel = (Wbmp.Format.BitsPerPixel + 7) / 8;
unsafe
{
byte* pImgData = (byte*)Wbmp.BackBuffer;
// set alpha to transparent for any pixel with red < 0x88 and invert others
int cRowStart = 0;
int cColStart = 0;
for (int row = 0; row < height; row++)
{
cColStart = cRowStart;
for (int col = 0; col < width; col++)
{
byte* bPixel = pImgData + cColStart;
UInt32* iPixel = (UInt32*)bPixel;
if (bPixel[2 /* bgRa */] < 0x44)
{
// set to 50% transparent
bPixel[3 /* bgrA */] = 0x7f;
}
else
{
// invert but maintain alpha
*iPixel = *iPixel ^ 0x00ffffff;
}
cColStart += bytesPerPixel;
}
cRowStart += stride;
}
}
}
And the routine of using it is like this:
masterImage.Lock();
ConvertImage(ref masterImage);
masterImage.AddDirtyRect(new Int32Rect(0, 0, masterImage.PixelWidth, masterImage.PixelHeight));
masterImage.Unlock();

Nonspecific exception when running unsafe code to process images

I'm doing some image processing and ran in to an exception.
Let me explain the logic process;
Resize the image to to a smaller size
Turn it grayscale
Threshold the image
Save it for use later on.
When you threshold the image, the constructor can take an int that sets the intensity of the filter. The best way I've found to get this "magic number" is to use a method called GetOtsuThreshold. It uses unsafe code but works well. However, something strange happens when you call that method. After you call the otsu method, it causes the Aforge...Threshold.ApplyInPlace() method to throw a Parameter is not valid exception. If you don't call it (When the code is commented out) the whole thing runs just fine though.
Wot's the deal?
EDIT: Found the problem; You must put a new a new image into the otsu method because it disposes of the image!!
using System;
using System.Drawing;
using System.Drawing.Imaging;
using AForge.Imaging.Filters;
namespace Puma.Ocr.Tests
{
class FormatImage
{
public static Bitmap _FullImageOfCoin;
public FormatImage(string path)
{
_FullImageOfCoin = ScaleImage(new Bitmap(path), 2000, 2000);
GrayscaleImage();
ThresholdImage();
}
private void GrayscaleImage()
{
Grayscale filter = new Grayscale(0.2125, 0.7154, 0.0721);
// apply the filter
_FullImageOfCoin = filter.Apply(_FullImageOfCoin);
}
private void ThresholdImage()
{
//Causes the exception
Threshold threshold = new Threshold(getOtsuThreshold(_FullImageOfCoin));
//Runs fine
//Threshold threshold = new Threshold();
threshold.ApplyInPlace(_FullImageOfCoin);
_FullImageOfCoin.Save(#"C:\users\school\desktop\thresholded.bmp");
}
public static Bitmap ScaleImage(Bitmap image, int maxWidth, int maxHeight)
{
var ratioX = (double)maxWidth / image.Width;
var ratioY = (double)maxHeight / image.Height;
var ratio = Math.Min(ratioX, ratioY);
var newWidth = (int)(image.Width * ratio);
var newHeight = (int)(image.Height * ratio);
var newImage = new Bitmap(newWidth, newHeight);
Graphics.FromImage(newImage).DrawImage(image, 0, 0, newWidth, newHeight);
return newImage;
}
public int getOtsuThreshold(Bitmap bmp)
{
byte t = 0;
float[] vet = new float[256];
int[] hist = new int[256];
vet.Initialize();
float p1, p2, p12;
int k;
BitmapData bmData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
unsafe
{
byte* p = (byte*)(void*)bmData.Scan0.ToPointer();
getHistogram(p, bmp.Width, bmp.Height, bmData.Stride, hist);
for (k = 1; k != 255; k++)
{
p1 = Px(0, k, hist);
p2 = Px(k + 1, 255, hist);
p12 = p1 * p2;
if (p12 == 0)
p12 = 1;
float diff = (Mx(0, k, hist) * p2) - (Mx(k + 1, 255, hist) * p1);
vet[k] = (float)diff * diff / p12;
}
}
bmp.UnlockBits(bmData);
t = (byte)findMax(vet, 256);
bmp.Dispose();
return t;
}
private unsafe void getHistogram(byte* p, int w, int h, int ws, int[] hist)
{
hist.Initialize();
for (int i = 0; i < h; i++)
{
for (int j = 0; j < w * 3; j += 3)
{
int index = i * ws + j;
hist[p[index]]++;
}
}
}
private int findMax(float[] vec, int n)
{
float maxVec = 0;
int idx = 0;
int i;
for (i = 1; i <= n - 1; i++)
{
if (vec[i] > maxVec)
{
maxVec = vec[i];
idx = i;
}
}
return idx;
}
private float Px(int init, int end, int[] hist)
{
int sum = 0;
int i;
for (i = init; i <= end; i++)
sum += hist[i];
return (float)sum;
}
// function is used to compute the mean values in the equation (mu)
private float Mx(int init, int end, int[] hist)
{
int sum = 0;
int i;
for (i = init; i <= end; i++)
sum += i * hist[i];
return (float)sum;
}
}
}

A few ideas:
Use debugging and follow this method step by step to see if it is well working
Your getOtsuThreshold(Bitmap bmp) is returning an int, but the variable t returned is a byte: try to cast the value?
If the int returned by getOtsuThreshold is okay, check that the value is in the good range provided the API info (http://www.aforgenet.com/framework/docs/html/503a43b9-d98b-a19f-b74e-44767916ad65.htm):
Since the filter can be applied as to 8 bpp and to 16 bpp images, the
ThresholdValue value should be set appropriately to the pixel format.
In the case of 8 bpp images the threshold value is in the [0, 255]
range, but in the case of 16 bpp images the threshold value is in the
[0, 65535] range.

Alright, found the answer. By putting the _Fullimage of coin into the otsu method it stripped the variable of all it properties. I don't know how, but by putting a new Bitmap into the otsu method it fixed the problem.

I have a function that creates histogram of each Bitmap. How can I create another 3 histograms for R.G.B of each Bitmap?

This is the histogram function I'm using today and if I'm not wrong it's creating an histogram by Gray color.
What i want is another function that will return me 3 histograms of each Bitmap:
The first histogram will be of the Red color of the bitmap the second for the Green color and the last one for the Blue color.
public static long[] GetHistogram(Bitmap b)
{
long[] myHistogram = new long[256];
BitmapData bmData = null;
try
{
//Lock it fixed with 32bpp
bmData = b.LockBits(new Rectangle(0, 0, b.Width, b.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
int scanline = bmData.Stride;
System.IntPtr Scan0 = bmData.Scan0;
unsafe
{
byte* p = (byte*)(void*)Scan0;
int nWidth = b.Width;
int nHeight = b.Height;
for (int y = 0; y < nHeight; y++)
{
for (int x = 0; x < nWidth; x++)
{
long Temp = 0;
Temp += p[0]; // p[0] - blue, p[1] - green , p[2]-red
Temp += p[1];
Temp += p[2];
Temp = (int)Temp / 3;
myHistogram[Temp]++;
//we do not need to use any offset, we always can increment by pixelsize when
//locking in 32bppArgb - mode
p += 4;
}
}
}
b.UnlockBits(bmData);
}
catch
{
try
{
b.UnlockBits(bmData);
}
catch
{
}
}
return myHistogram;
}
How may I do it ?

In the part where you specify
Temp += p[0]
...
put the three values into separate histograms:
histB[p[0]]++;
histG[p[1]]++;
histR[p[2]]++;

You can use jagged array(array of arrays) where values from p[0],p[1],p[2] may put into the jagged array. and then work with the indexed values of the jagged array.
Hope this helps

C# LockBits perfomance (int[,] to byte[])

Graphics g;
using (var bmp = new Bitmap(_frame, _height, PixelFormat.Format24bppRgb))
{
var data = bmp.LockBits(new Rectangle(0, 0, _frame, _height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
var bmpWidth = data.Stride;
var bytes = bmpWidth * _height;
var rgb = new byte[bytes];
var ptr = data.Scan0;
Marshal.Copy(ptr, rgb, 0, bytes);
for (var i = 0; i < _frame; i++)
{
var i3 = (i << 1) + i;
for (var j = 0; j < _height; j++)
{
var ij = j * bmpWidth + i3;
var val = (byte)(_values[i, j]);
rgb[ij] = val;
rgb[ij + 1] = val;
rgb[ij + 2] = val;
}
}
Marshal.Copy(rgb, 0, ptr, bytes);
bmp.UnlockBits(data);
g = _box.CreateGraphics();
g.InterpolationMode = InterpolationMode.NearestNeighbor;
g.DrawImage(bmp, 0, 0, _box.Width, _box.Height);
}
g.Dispose();
I use this code to convert an array of RGB values ​​(grayscale) in the PictureBox, but it's slow. Please tell me my mistakes.
At the moment, an array of 441 000 items handled for 35 ms.
I need to handle an array of 4 million for the same time.

You can skip the first Array.Copy where you copy the data from the image to the array, as you will be overwriting all the data in the array anyway.
That will shave off something like 25% of time, but if you want it faster you will have to use an unsafe code block so that you can use pointers. That way you can get around the range checking when you access arrays, and you can write the data directly into the image data instead of copying it.

I totally agree with Guffa's answer. Using an unsafe code block will speed up things.
To further improve performance, you could execute your for loop in parallel by using the Parallel class in the .Net framework. For large bitmaps this improves performance.
Here is a small code sample:
using (Bitmap bmp = (Bitmap)Image.FromFile(#"mybitmap.bmp"))
{
int width = bmp.Width;
int height = bmp.Height;
BitmapData bd = bmp.LockBits(new Rectangle(0, 0, width, height),
System.Drawing.Imaging.ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
byte* s0 = (byte*)bd.Scan0.ToPointer();
int stride = bd.Stride;
Parallel.For(0, height, (y1) =>
{
int posY = y1*stride;
byte* cpp = s0 + posY;
for (int x = 0; x < width; x++)
{
// Set your pixel values here.
cpp[0] = 255;
cpp[1] = 255;
cpp[2] = 255;
cpp += 3;
}
});
bmp.UnlockBits(bd);
}
To keep the example simple I've set the pixel values to a constant value. Note, to compile the example above you have to allow unsafe code.
Hope, this helps.

In addition to Guffa's excellent advice, I would suggest that you profile your code to see where it's taking the time. Be sure that when you're timing this, you are running in release mode without the debugger attached.
I wouldn't be surprised if the call to DrawImage is taking up most of the time. You're scaling the image there, which can be pretty expensive. How large is the box that you're drawing the image to?
Finally, although this won't affect performance, you should change your code to read:
using (Graphics g = _box.CreateGraphics())
{
g.InterpolationMode = InterpolationMode.NearestNeighbor;
g.DrawImage(bmp, 0, 0, _box.Width, _box.Height);
}
And get rid of the first and last lines in your example.

Try this using unsafe code:
byte* rp0;
int* vp0;
fixed (byte* rp1 = rgb)
{
rp0 = rp1;
fixed (int* vp1 = _values)
{
vp0 = vp1;
Parallel.For(0, _width, (i) =>
{
var val = (byte)vp0[i];
rp0[i] = val;
rp0[i + 1] = val;
rp0[i + 2] = val;
});
}
}
Runs very fast for me

My understanding is that multidimentional (square) arrays are pretty slow in .Net. You might try changing your _values array to be a single dimension array instead.
Here is one reference, there are many more if you search:
http://odetocode.com/articles/253.aspx
Array perf example.
using System;
using System.Diagnostics;
class Program
{
static void Main(string[] args)
{
int w = 1000;
int h = 1000;
int c = 1000;
TestL(w, h);
TestM(w, h);
var swl = Stopwatch.StartNew();
for (int i = 0; i < c; i++)
{
TestL(w, h);
}
swl.Stop();
var swm = Stopwatch.StartNew();
for (int i = 0; i < c; i++)
{
TestM(w, h);
}
swm.Stop();
Console.WriteLine(swl.Elapsed);
Console.WriteLine(swm.Elapsed);
Console.ReadLine();
}
static void TestL(int w, int h)
{
byte[] b = new byte[w * h];
int q = 0;
for (int x = 0; x < w; x++)
for (int y = 0; y < h; y++)
b[q++] = 1;
}
static void TestM(int w, int h)
{
byte[,] b = new byte[w, h];
for (int y = 0; y < h; y++)
for (int x = 0; x < w; x++)
b[y, x] = 1;
}
}