Addendum: it seems to run correctly when I uncheck "optimize code" which leads me to believe it is some quirky configuration problem
Firstly I am trying to run unmanaged code. I have "allow unsafe code" checked. It is pointing to this line of code here where I am trying to read a bitmap without using the relatively slow getpixel:
byte[] buff = { scanline[xo], scanline[xo + 1], scanline[xo + 2], 0xff };
Entire snippet is below. How can I correct this problem?
private const int PIXELSIZE = 4; // Number of bytes in a pixel
BitmapData mainImageData = mainImage.LockBits(new Rectangle(0, 0, mainImage.Width, mainImage.Height), ImageLockMode.ReadOnly, mainImage.PixelFormat);
List<Point> results = new List<Point>();
foundRects = new List<Rectangle>();
for (int y = 0; y < mainImageData.Height
{
byte* scanline = (byte*)mainImageData.Scan0 + (y * mainImageData.Stride);
for (int x = 0; x < mainImageData.Width; x++)
{
int xo = x * PIXELSIZE;
byte[] buff = { scanline[xo], scanline[xo + 1],
scanline[xo + 2], 0xff };
int val = BitConverter.ToInt32(buff, 0);
// Pixle value from subimage in desktop image
if (pixels.ContainsKey(val) && NotFound(x, y))
{
Point loc = (Point)pixels[val];
int sx = x - loc.X;
int sy = y - loc.Y;
// Subimage occurs in desktop image
if (ImageThere(mainImageData, subImage, sx, sy))
{
Point p = new Point(x - loc.X, y - loc.Y);
results.Add(p);
foundRects.Add(new Rectangle(x, y, subImage.Width,
subImage.Height));
}
}
}
It's hard to say with the limited information we have, but I see a couple of obvious problems, one of which addresses your issue directly:
You are not checking the pixel format, but are assuming that it is 32bppRGB. It is likely 24bppRGB, and that would explain the error.
You are reading the RGB values incorrectly; Windows internally stores bitmaps in BGR order.
You are not calling UnlockBits at the end of the method.
What are the values xo + 1 and xo + 2 when x is mainImageData.Width - 1? They are surely walking off the reservation.
The most likely reason is that the image isn't 32bpp. it could be 24bpp, or possibly even 8 or 16 bit.
You should be getting PIXELSIZE from PixelFormat Rather than hard-coding it to 4.
Check to see that the value you are getting for Stride agrees with the Width.
Related
I have a problem. I need to perform this function with lockbits. Please I need help.
public void xPix(Bitmap bmp, int n, Color cx, Color nx)
{
try
{
for (int y = 0; y < bmp.Height; y++)
{
for (int x = 0; x < bmp.Width; x += (n * 2))
{
cx = bmp.GetPixel(x, y);
if (x + n <= bmp.Width - 1) nx = bmp.GetPixel(x + n, y);
bmp.SetPixel(x, y, nx);
if (x + n <= bmp.Width - 1) bmp.SetPixel(x + n, y, cx);
}
}
}
catch { }
}
There were lots of things that didn't make sense to me about your code. I fixed the pieces that were preventing an image from appearing and here is the result. I will explain my changes after the code.
public void xPix(Bitmap bmp, int n, Color cx, Color nx)
{
var img = bmp.LockBits(new Rectangle(Point.Empty, bmp.Size), System.Drawing.Imaging.ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
byte[] bmpBytes = new byte[Math.Abs(img.Stride) * img.Height];
System.Runtime.InteropServices.Marshal.Copy(img.Scan0, bmpBytes, 0, bmpBytes.Length);
for (int y = 0; y < img.Height; y++)
{
for (int x = 0; x < img.Width; x+=n*2)
{
cx = Color.FromArgb(BitConverter.ToInt32(bmpBytes, y * Math.Abs(img.Stride) + x * 4));
if (x + n <= img.Width - 1) nx = Color.FromArgb(BitConverter.ToInt32(bmpBytes, y * Math.Abs(img.Stride) + x * 4));
BitConverter.GetBytes(nx.ToArgb()).CopyTo(bmpBytes, y * Math.Abs(img.Stride) + x * 4);
if (x + n <= img.Width - 1) BitConverter.GetBytes(cx.ToArgb()).CopyTo(bmpBytes, y * Math.Abs(img.Stride) + (x + n) * 4);
}
}
System.Runtime.InteropServices.Marshal.Copy(bmpBytes, 0, img.Scan0, bmpBytes.Length);
bmp.UnlockBits(img);
}
protected override void OnClick(EventArgs e)
{
base.OnClick(e);
Bitmap bmp = new Bitmap(#"C:\Users\bluem\Downloads\Default.png");
for (int i = 0; i < bmp.Width; i++)
{
xPix(bmp, new Random().Next(20) + 1, System.Drawing.Color.White, System.Drawing.Color.Green);
}
Canvas.Image = bmp;
}
There's no such class as LockBitmap so I replaced it with the result of a call to Bitmap.LockBits directly.
The result of LockBits does not include functions for GetPixel and SetPixel, so I did what one normally does with the result of LockBits (see https://learn.microsoft.com/en-us/dotnet/api/system.drawing.bitmap.lockbits?view=netframework-4.7.2) and copied the data into a byte array instead.
When accessing the byte data directly, some math must be done to convert the x and y coordinates into a 1-dimensional coordinate within the array of bytes, which I did.
When accessing the byte data directly under the System.Drawing.Imaging.PixelFormat.Format32bppArgb pixel format, multiple bytes must be accessed to convert between byte data and a pixel color, which I did with BitConverter.GetBytes, BitConverter.ToInt32, Color.FromArgb and Color.ToArgb.
I don't think it's a good idea to be changing the Image in the middle of painting it. You should either be drawing the image directly during the Paint event, or changing the image outside the Paint event and allowing the system to draw it. So I used the OnClick of my form to trigger the function instead.
The first random number I got was 0, so I had to add 1 to avoid an endless loop.
The cx and nx parameters never seem to be used as inputs, so I put arbitrary color values in for them. Your x and y variables were not defined/declared anywhere.
If you want faster on-image-action, you can use Marshall.Copy method with Parallel.For
Why dont use GetPixel method? Because every time you call it, your ALL image is loaded to memory. GetPixel get one pixel, and UNLOAD all image. And in every iteration, ALL image is loaded to memory (for example, if u r working on 500x500 pix image, GetPixel will load 500x500 times whole pixels to memory). When you work on images with C# (CV stuff), work on raw bytes from memory.
I will show how to use with Lockbits in Binarization because its easy to explain.
int pixelBPP = Image.GetPixelFormatSize(resultBmp.PixelFormat) / 8;
unsafe
{
BitmapData bmpData = resultBmp.LockBits(new Rectangle(0, 0, resultBmp.Width, resultBmp.Height), ImageLockMode.ReadWrite, resultBmp.PixelFormat);
byte* ptr = (byte*)bmpData.Scan0; //addres of first line
int height = resultBmp.Height;
int width = resultBmp.Width * pixelBPP;
Parallel.For(0, height, y =>
{
byte* offset = ptr + (y * bmpData.Stride); //set row
for(int x = 0; x < width; x = x + pixelBPP)
{
byte value = (offset[x] + offset[x + 1] + offset[x + 2]) / 3 > threshold ? Byte.MaxValue : Byte.MinValue;
offset[x] = value;
offset[x + 1] = value;
offset[x + 2] = value;
if (pixelBPP == 4)
{
offset[x + 3] = 255;
}
}
});
resultBmp.UnlockBits(bmpData);
}
Now, example with Marshall.copy:
BitmapData bmpData = resultBmp.LockBits(new Rectangle(0, 0, resultBmp.Width, resultBmp.Height),
ImageLockMode.ReadWrite,
resultBmp.PixelFormat
);
int bytes = bmpData.Stride * resultBmp.Height;
byte[] pixels = new byte[bytes];
Marshal.Copy(bmpData.Scan0, pixels, 0, bytes); //loading bytes to memory
int height = resultBmp.Height;
int width = resultBmp.Width;
Parallel.For(0, height - 1, y => //seting 2s and 3s
{
int offset = y * stride; //row
for (int x = 0; x < width - 1; x++)
{
int positionOfPixel = x + offset + pixelFormat; //remember about pixel format!
//do what you want with pixel
}
}
});
Marshal.Copy(pixels, 0, bmpData.Scan0, bytes); //copying bytes to bitmap
resultBmp.UnlockBits(bmpData);
Remember, when you warking with RAW bytes very important is to remember about PixelFormat. If you work on RGBA image, you need to set up every channel. (for example offset + x + pixelFormat). I showed it in Binarization example, how to deak with RGBA image with raw data. If lockbits are not fast enough, use Marshall.Copy
Well, I have a code to apply a Rain Bow filter in "x" image, I have to do in two ways: Sequential & parallel, my sequential code is working without problems, but the parallel section doesn't work. And I have no idea, why?.
Code
public static Bitmap RainbowFilterParallel(Bitmap bmp)
{
Bitmap temp = new Bitmap(bmp.Width, bmp.Height);
int raz = bmp.Height / 4;
Parallel.For(0, bmp.Width, i =>
{
Parallel.For(0, bmp.Height, x =>
{
if (i < (raz))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R / 5, bmp.GetPixel(i, x).G, bmp.GetPixel(i, x).B));
}
else if (i < (raz * 2))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R, bmp.GetPixel(i, x).G / 5, bmp.GetPixel(i, x).B));
}
else if (i < (raz * 3))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R, bmp.GetPixel(i, x).G, bmp.GetPixel(i, x).B / 5));
}
else if (i < (raz * 4))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R / 5, bmp.GetPixel(i, x).G, bmp.GetPixel(i, x).B / 5));
}
else
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R / 5, bmp.GetPixel(i, x).G / 5, bmp.GetPixel(i, x).B / 5));
}
});
});
return temp;
}
Besides, In a moments the program return the same error but says "The object is already in use".
PS. I'm beginner with c#, and I Searched this topic in another post and I found nothing.
Thank you very much in advance
As commenter Ron Beyer points out, using the SetPixel() and GetPixel() methods is very slow. Each call to one of those methods involves a lot of overhead in the transition between your managed code down to the actual binary buffer that the Bitmap object represents. There are a lot of layers there, and the video driver typically gets involved which requires transitions between user and kernel level execution.
But besides being slow, these methods also make the object "busy", and so if an attempt to use the bitmap (including calling one of those methods) is made between the time one of those methods is called and when it returns (i.e. while the call is in progress), an error occurs with the exception you saw.
Since the only way that parallelizing your current code would be helpful is if these method calls could occur concurrently, and since they simply cannot, this approach isn't going to work.
On the other hand, using the LockBits() method is not only guaranteed to work, there's a very good chance that you will find the performance is so much better using LockBits() that you don't even need to parallelize the algorithm. But should you decide you do, because of the way LockBits() works — you gain access to a raw buffer of bytes that represents the bitmap image — you can easily parallelize the algorithm and take advantage of multiple CPU cores (if present).
Note that when using LockBits() you will be working with the Bitmap object at a level that you might not be accustomed to. If you are not already knowledgeable with how bitmaps really work "under the hood", you will have to familiarize yourself with the way that bitmaps are actually stored in memory. This includes understanding what the different pixel formats mean, how to interpret and modify pixels for a given format, and how a bitmap is laid out in memory (e.g. the order of rows, which can vary depending on the bitmap, as well as the "stride" of the bitmap).
These things are not terribly hard to learn, but it will require patience. It is well worth the effort though, if performance is your goal.
Parallel is hard on the singular mind. And mixing it with legacy GDI+ code can lead to strange results..
Your code has numerous issues:
You call GetPixel three times per pixel instead of once
You are accessing the pixels not horizontally as you should
You call y x and x i; the machine won't mind but us people do
You are using way too much parallelization. No use to have much more of it than you have cores. It creates overhead that is bound to eat up any gains, unless your inner loop has a really hard job to do, like millions of calculations..
But the exception you get has nothing to do with these issues. And one mistake you don't make is to access the same pixel in parallel... So why the crash?
After cleaning up the code I found that the error in the stack trace pointed to SetPixel and there to System.Drawing.Image.get_Width(). The former is obvious, the latter not part of our code..!?
So I dug into the source code at referencesource.microsoft.com and found this:
/// <include file='doc\Bitmap.uex' path='docs/doc[#for="Bitmap.SetPixel"]/*' />
/// <devdoc>
/// <para>
/// Sets the color of the specified pixel in this <see cref='System.Drawing.Bitmap'/> .
/// </para>
/// </devdoc>
public void SetPixel(int x, int y, Color color) {
if ((PixelFormat & PixelFormat.Indexed) != 0) {
throw new InvalidOperationException(SR.GetString(SR.GdiplusCannotSetPixelFromIndexedPixelFormat));
}
if (x < 0 || x >= Width) {
throw new ArgumentOutOfRangeException("x", SR.GetString(SR.ValidRangeX));
}
if (y < 0 || y >= Height) {
throw new ArgumentOutOfRangeException("y", SR.GetString(SR.ValidRangeY));
}
int status = SafeNativeMethods.Gdip.GdipBitmapSetPixel(new HandleRef(this, nativeImage), x, y, color.ToArgb());
if (status != SafeNativeMethods.Gdip.Ok)
throw SafeNativeMethods.Gdip.StatusException(status);
}
The real work is done by SafeNativeMethods.Gdip.GdipBitmapSetPixel but before that the method does a bounds check on the Bitmap's Width and Height. And while these in our case of course never change the system still won't allow accessing them in parallel and hence crashes when at some point the checks are happening interwoven. Totally uncesessary, of course, but there you go..
So GetPixel (which has the same behaviour) and SetPixel can't safely be used in a parallel processing.
Two ways out of it:
We can add locks to the code and thus make sure the checks won't happen at the 'same' time:
public static Bitmap RainbowFilterParallel(Bitmap bmp)
{
Bitmap temp = new Bitmap(bmp);
int raz = bmp.Height / 4;
int height = bmp.Height;
int width = bmp.Width;
// set a limit to parallesim
int maxCore = 7;
int blockH = height / maxCore + 1;
//lock (temp)
Parallel.For(0, maxCore, cor =>
{
//Parallel.For(0, bmp.Height, x =>
for (int yb = 0; yb < blockH; yb++)
{
int i = cor * blockH + yb;
if (i >= height) continue;
for (int x = 0; x < width; x++)
{
{
Color c;
// lock the Bitmap just for the GetPixel:
lock (temp) c = temp.GetPixel(x, i);
byte R = c.R;
byte G = c.G;
byte B = c.B;
if (i < (raz)) { R = (byte)(c.R / 5); }
else if (i < raz + raz) { G = (byte)(c.G / 5); }
else if (i < raz * 3) { B = (byte)(c.B / 5); }
else if (i < raz * 4) { R = (byte)(c.R / 5); B = (byte)(c.B / 5); }
else { G = (byte)(c.G / 5); R = (byte)(c.R / 5); }
// lock the Bitmap just for the SetPixel:
lock (temp) temp.SetPixel(x, i, Color.FromArgb(R,G,B));
};
}
};
});
return temp;
}
Note that limiting parallism is so important there is even a member in the ParallelOptions class and a parameter inParallel.For to control it! I have set the maximum core numer to 7, but this would be better:
int degreeOfParallelism = Environment.ProcessorCount - 1;
So this should save us some overhead. But still: I'd expect that to be slower than a corrected sequential method!
Instead going for a LockBits as Peter and Ron have suggested method makes things really fast (1ox) and adding parallelism potentially even faster still..
So finally to finish up this length answer, here is a Lockbits plus Limited-Parallel solution:
public static Bitmap RainbowFilterParallelLockbits(Bitmap bmp)
{
Bitmap temp = null;
temp = new Bitmap(bmp);
int raz = bmp.Height / 4;
int height = bmp.Height;
int width = bmp.Width;
Rectangle rect = new Rectangle(Point.Empty, bmp.Size);
BitmapData bmpData = temp.LockBits(rect,ImageLockMode.ReadOnly, temp.PixelFormat);
int bpp = (temp.PixelFormat == PixelFormat.Format32bppArgb) ? 4 : 3;
int size = bmpData.Stride * bmpData.Height;
byte[] data = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(bmpData.Scan0, data, 0, size);
var options = new ParallelOptions();
int maxCore = Environment.ProcessorCount - 1;
options.MaxDegreeOfParallelism = maxCore > 0 ? maxCore : 1;
Parallel.For(0, height, options, y =>
{
for (int x = 0; x < width; x++)
{
{
int index = y * bmpData.Stride + x * bpp;
if (y < (raz)) data[index + 2] = (byte) (data[index + 2] / 5);
else if (y < (raz * 2)) data[index + 1] = (byte)(data[index + 1] / 5);
else if (y < (raz * 3)) data[index ] = (byte)(data[index ] / 5);
else if (y < (raz * 4))
{ data[index + 2] = (byte)(data[index + 2] / 5);
data[index] = (byte)(data[index] / 5); }
else
{ data[index + 2] = (byte)(data[index + 2] / 5);
data[index + 1] = (byte)(data[index + 1] / 5);
data[index] = (byte)(data[index] / 5); }
};
};
});
System.Runtime.InteropServices.Marshal.Copy(data, 0, bmpData.Scan0, data.Length);
temp.UnlockBits(bmpData);
return temp;
}
While not strictly relevant I wanted to post a better faster version than any of the one's that I see given in the answers. This is the fastest way I know of to iterate through a bitmap and save the results in C#. In my work we need to go through millions of large images, this is just me grabbing the red channel and saving it for my own purposes but it should give you the idea of how to work
//Parallel Unsafe, Corrected Channel, Corrected Standard div 5x faster
private void TakeApart_Much_Faster(Bitmap processedBitmap)
{
_RedMin = byte.MaxValue;
_RedMax = byte.MinValue;
_arr = new byte[BMP.Width, BMP.Height];
long Sum = 0,
SumSq = 0;
BitmapData bitmapData = processedBitmap.LockBits(new Rectangle(0, 0, processedBitmap.Width, processedBitmap.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
//this is a much more useful datastructure than the array but it's slower to fill.
points = new ConcurrentDictionary<Point, byte>();
unsafe
{
int bytesPerPixel = Image.GetPixelFormatSize(bitmapData.PixelFormat) / 8;
int heightInPixels = bitmapData.Height;
int widthInBytes = bitmapData.Width * bytesPerPixel;
_RedMin = byte.MaxValue;
_RedMax = byte.MinValue;
byte* PtrFirstPixel = (byte*)bitmapData.Scan0;
Parallel.For(0, heightInPixels, y =>
{
//pointer to the first pixel so we don't lose track of where we are
byte* currentLine = PtrFirstPixel + (y * bitmapData.Stride);
for (int x = 0; x < widthInBytes; x = x + bytesPerPixel)
{
//0+2 is red channel
byte redPixel = currentLine[x + 2];
Interlocked.Add(ref Sum, redPixel);
Interlocked.Add(ref SumSq, redPixel * redPixel);
//divide by three since we are skipping ahead 3 at a time.
_arr[x/3, y] = redPixel;
_RedMin = redPixel < _RedMin ? _RedMin : redPixel;
_RedMax = redPixel > RedMax ? RedMax : redPixel;
}
});
_RedMean = Sum / TotalPixels;
_RedStDev = Math.Sqrt((SumSq / TotalPixels) - (_RedMean * _RedMean));
processedBitmap.UnlockBits(bitmapData);
}
}
I'm trying to make a program that, using pointers, detects lines in an image and removes those lines. Currently, the detecting lines part is working really well, and for the most part the removing the lines part is working as well. However, after around 150-200 images, the program will throw random AccessViolationExceptions in places that have nothing to do with the unsafe bits of the code.
This is the bit that does the line removal:
static unsafe Bitmap RemoveLines(Bitmap input, int[] horizontalLines, int[] verticalLines)
{
Bitmap output;
if (input.PixelFormat == PixelFormat.Format24bppRgb)
{
output = (Bitmap) input.Clone();
}
else
{
output = ConvertTo24bpp((Bitmap)input.Clone());
}
BitmapData bitmapData = output.LockBits(new Rectangle(0, 0, output.Width, output.Height), ImageLockMode.ReadWrite, output.PixelFormat);
int w = output.Width;
int h = output.Height;
int bpp = 3;
int s = bitmapData.Stride;
byte* p = (byte*) bitmapData.Scan0;
for (int r = 0; r < h; r++)
{
for (int c = 0; c < h; c++)
{
if (horizontalLines.Contains(r) || verticalLines.Contains(c))
{
int i = (r * s) + c * bpp;
p[i + 0] = 255;
p[i + 1] = 255;
p[i + 2] = 255;
}
}
}
output.UnlockBits(bitmapData);
return output;
}
After this code, I save the resulting Bitmap as well as embedding it in another Bitmap for comparison purposes:
// ... Detect lines and such
Bitmap export = new Bitmap(bitmap.Width * 3, bitmap.Height, PixelFormat.Format24bppRgb);
Graphics fg = Graphics.FromImage(export);
fg.DrawImage(bitmap, 0, 0); // Draw the original input bitmap
fg.DrawImage(edited, bitmap.Width, 0); // Draw the input after processing (Line Detection)
try
{
Bitmap lineRemoved = RemoveLines(bitmap, horizontalLines.ToArray(), verticalLines.ToArray()); // Remove lines based on earlier detection
lineRemoved.Save(cellDirectory + "\\Lines\\cell_lr_" + i.ToString("D2") + j.ToString("D2") + ".gif", ImageFormat.Gif); // Save image after removal
fg.DrawImage(lineRemoved, bitmap.Width * 2, 0); // Add image to composite for comparison; This line is what throws the error most of the time
lineRemoved.Dispose();
export.Save(cellDirectory + "\\Lines\\cell" + i.ToString("D2") + j.ToString("D2") + ".gif", ImageFormat.Gif);
}
catch (Exception ex)
{ }
The DrawImage call is what throws errors, and it is always an AccessViolationException followed by an InvalidOperationException. Looking at lineRemoved during the error shows that most of its members have "threw exception of type 'InvalidOperationException'" rather than actual values, even though one line before the same Bitmap saved just fine on its own. The input bitmap remains unchanged throughout the code, and is always Cloned or drawn to a different bitmap when I need to alter it in any way.
I've tried commenting out the lines after saving lineRemoved, but then the same error pops up later in the code. What's more, that try/catch doesn't actually catch the Exception - it always says unhandled. It's got to be something to do with the pointers, but otherwise I am completely lost as to what is causing this.
Your code contains a subtle one-character bug. The line that reads
for (int c = 0; c < h; c++)
should be
for (int c = 0; c < w; c++)
If the image is in landscape orientation, your bug would cause the right part of the image not being processed.
If the image is in protrait orientation, it would cause the buffer to overflow, leading to an access violation exception (if you're lucky) or memory corruption (if you're not).
That being said, your algorithm is not very efficient. For example, you are doing the calculation
int i = (r * s) + c * bpp;
for every pixel you're drawing, while obviously (r * s) doesn't change in the inner loop, and c * bpp can be replaced by something like currentPixel += bpp.
In fact it would probably be more efficient to loop over horizontalLines and verticalLines.
I am trying to copy camera metadata into a Bitmap, and seing as each value in the metadata is a 16bit (or ushort) I thought it would be sensible to display it in a 16bpp garyscale Bitmap. The code I wrote is as follows:
// Getting the metadata from the device
metaData = new DepthMetaData();
dataSource.GetMetaData(metaData);
// Setting up bitmap, rect and data to use pointer
Bitmap bitmap = new Bitmap(metaData.XRes, metaData.YRes, PixelFormat.Format16bppGrayScale);
Rectangle rect = new Rectangle(0, 0, bitmap.Width, bitmap.Height);
BitmapData data = bitmap.LockBits(rect, ImageLockMode.WriteOnly, PixelFormat.Format16bppGrayScale);
// Pointer pointing to metadata
ushort* ptrMetaData = (ushort*)dataSource.DepthMapPtr.ToPointer();
lock(this)
{
// Runs through the whole bitmap and assigns the entry in the metadata
// to a pixel
for (int y = 0; y < bitmap.Height; ++y)
{
ushort* ptrDestination = (ushort*)data.Scan0.ToPointer() + y * data.Stride;
for (int x = 0; x < bitmap.Width; ++x, ++ptrMetaData)
{
ptrDestination[x] = (ushort)*ptrMetaData;
}
}
}
// Once done unlock the bitmap so that it can be read again
bitmap.UnlockBits(data);
When running the Metadata's XRes = 640 and YRes = 480. The code throws a memory access exception in the for-loops on "ptrDestination[x] = (ushort)*ptrMetaData;" after only running though 240, half the total, lines.
I used this with 8bpp where I reduced the resolution and it worked nicely, so I don't see why it should not here. Maybe someone finds the problem.
Thanks already
ushort* ptrDestination = (ushort*)data.Scan0.ToPointer() + y * data.Stride;
The data.Stride value is expressed in bytes, not ushorts. So the pointer is off by a factor of 2 so it bombs at bitmap.Height/2. Your for loops are broken, swap bitmap.Width and bitmap.Height. The lock keyword doesn't make much sense here, you are accessing thread-local data, other than dataSource. Fix:
for (int y = 0; y < bitmap.Height; ++y)
{
ushort* ptrDestination = (ushort*)data.Scan0.ToPointer() + y * data.Stride / 2;
for (int x = 0; x < bitmap.Width; ++x, ++ptrMetaData)
{
ptrDestination[x] = (ushort)*ptrMetaData;
}
}
I'd like to find an image (needle) within an image (haystack).
To keep things simple I take two screenshots of my desktop. One full size (haystack) and a tiny one (needle). I then loop through the haystack image and try to find the needle image.
capture needle and haystack screenshot
loop through haystack, looking out for haystack[i] == first pixel of needle
[if 2. is true:] loop through the 2nd to last pixel of needle and compare it to haystack[i]
Expected result: the needle image is found at the correct location.
I already got it working for some coordinates/widths/heights (A).
But sometimes bits seem to be "off" and therefore no match is found (B).
What could I be doing wrong? Any suggestions are welcome. Thanks.
var needle_height = 25;
var needle_width = 25;
var haystack_height = 400;
var haystack_width = 500;
A. example input - match
var needle = screenshot(5, 3, needle_width, needle_height);
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);
B. example input - NO match
var needle = screenshot(5, 5, needle_width, needle_height);
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);
1. capture needle and haystack image
private int[] screenshot(int x, int y, int width, int height)
{
Bitmap bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb);
Graphics.FromImage(bmp).CopyFromScreen(x, y, 0, 0, bmp.Size);
var bmd = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly, bmp.PixelFormat);
var ptr = bmd.Scan0;
var bytes = bmd.Stride * bmp.Height / 4;
var result = new int[bytes];
Marshal.Copy(ptr, result, 0, bytes);
bmp.UnlockBits(bmd);
return result;
}
2. try to find a match
public Point findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int i = 0; i < haystack.Length; i++)
{
if (haystack[i] == firstpixel)
{
var y = i / haystack_height;
var x = i % haystack_width;
var matched = checkmatch(haystack, needle, x, y);
if (matched)
return (new Point(x,y));
}
}
return new Point();
}
3. verify full match
public bool checkmatch(int[] haystack, int[] needle, int startx, int starty)
{
for (int y = starty; y < starty + needle_height; y++)
{
for (int x = startx; x < startx + needle_width; x++)
{
int haystack_index = y * haystack_width + x;
int needle_index = (y - starty) * needle_width + x - startx;
if (haystack[haystack_index] != needle[needle_index])
return false;
}
}
return true;
}
Instead of making two screenshots of your desktop with a time interval between them, I would take a screenshot once and cut "needle" and "haystack" from those same bitmap source. Otherwise you have the risk of a change of your desktop contents between the two moments where the screenshots are taken.
EDIT: And when your problem still occurs after that, I would try to save the image to a file and try again with that file using your debugger, giving you a reproducible situation.
First, there is a problem with the findmatch loop. You shouldn't just use the haystack image as an array, because you need to subtract needle's width and height from right and bottom respectively:
public Point? findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int y = 0; y < haystack_height - needle_height; y++)
for (int x = 0; x < haystack_width - needle_width; x++)
{
if (haystack[y * haystack_width + x] == firstpixel)
{
var matched = checkmatch(haystack, needle, x, y);
if (matched)
return (new Point(x, y));
}
}
return null;
}
That should probably solve the problem. Also, keep in mind that there might be multiple matches. For example, if "needle" is a completely white rectangle portion of a window, there will most likely be many matches in the entire screen. If this is a possibility, modify your findmatch method to continue searching for results after the first one is found:
public IEnumerable<Point> FindMatches(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int y = 0; y < haystack_height - needle_height; y++)
for (int x = 0; x < haystack_width - needle_width; x++)
{
if (haystack[y * haystack_width + x] == firstpixel)
{
if (checkmatch(haystack, needle, x, y))
yield return (new Point(x, y));
}
}
}
Next, you need to keep a habit of manually disposing all objects which implement IDisposable, which you have created yourself. Bitmap and Graphics are such objects, meaning that your screenshot method needs to be modified to wrap those objects in using statements:
private int[] screenshot(int x, int y, int width, int height)
{
// dispose 'bmp' after use
using (var bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb))
{
// dispose 'g' after use
using (var g = Graphics.FromImage(bmp))
{
g.CopyFromScreen(x, y, 0, 0, bmp.Size);
var bmd = bmp.LockBits(
new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly,
bmp.PixelFormat);
var ptr = bmd.Scan0;
// as David pointed out, "bytes" might be
// a bit misleading name for a length of
// a 32-bit int array (so I've changed it to "len")
var len = bmd.Stride * bmp.Height / 4;
var result = new int[len];
Marshal.Copy(ptr, result, 0, len);
bmp.UnlockBits(bmd);
return result;
}
}
}
The rest of the code seems ok, with the remark that it won't be very efficient for certain inputs. For example, you might have a large solid color as your desktop's background, which might result in many checkmatch calls.
If performance is of interest to you, you might want to check different ways to speed up the search (something like a modified Rabin-Karp comes to mind, but I am sure there are some existing algorithms which ensure that invalid candidates are skipped immediately).
I don't think your equations for haystack_index or needle_index are correct. It looks like you take the Scan0 offset into account when you copy the bitmap data, but you need to use the bitmap's Stride when calculating the byte position.
Also, the Format32bppArgb format uses 4 bytes per pixel. It looks like you are assuming 1 byte per pixel.
Here's the site I used to help with those equations: https://web.archive.org/web/20141229164101/http://bobpowell.net/lockingbits.aspx
Format32BppArgb: Given X and Y coordinates, the address of the first element in the
pixel is Scan0+(y * stride)+(x*4). This Points to the blue byte. The
following three bytes contain the green, red and alpha bytes.
Here are the class reference with example code that works great for my C# application for finding needle in haystack for each frame from a USB camera, in year 2018... I believe Accord is mostly a bunch of C# wrappers for fast C++ code.
Also check out the C# wrapper for Microsoft C++ DirectShow that I use to search for needle within each frame from a USB camera