Recognizing image within image in C# - c#

I'd like to find an image (needle) within an image (haystack).
To keep things simple I take two screenshots of my desktop. One full size (haystack) and a tiny one (needle). I then loop through the haystack image and try to find the needle image.
capture needle and haystack screenshot
loop through haystack, looking out for haystack[i] == first pixel of needle
[if 2. is true:] loop through the 2nd to last pixel of needle and compare it to haystack[i]
Expected result: the needle image is found at the correct location.
I already got it working for some coordinates/widths/heights (A).
But sometimes bits seem to be "off" and therefore no match is found (B).
What could I be doing wrong? Any suggestions are welcome. Thanks.
var needle_height = 25;
var needle_width = 25;
var haystack_height = 400;
var haystack_width = 500;
A. example input - match
var needle = screenshot(5, 3, needle_width, needle_height);
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);
B. example input - NO match
var needle = screenshot(5, 5, needle_width, needle_height);
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);
1. capture needle and haystack image
private int[] screenshot(int x, int y, int width, int height)
{
Bitmap bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb);
Graphics.FromImage(bmp).CopyFromScreen(x, y, 0, 0, bmp.Size);
var bmd = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly, bmp.PixelFormat);
var ptr = bmd.Scan0;
var bytes = bmd.Stride * bmp.Height / 4;
var result = new int[bytes];
Marshal.Copy(ptr, result, 0, bytes);
bmp.UnlockBits(bmd);
return result;
}
2. try to find a match
public Point findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int i = 0; i < haystack.Length; i++)
{
if (haystack[i] == firstpixel)
{
var y = i / haystack_height;
var x = i % haystack_width;
var matched = checkmatch(haystack, needle, x, y);
if (matched)
return (new Point(x,y));
}
}
return new Point();
}
3. verify full match
public bool checkmatch(int[] haystack, int[] needle, int startx, int starty)
{
for (int y = starty; y < starty + needle_height; y++)
{
for (int x = startx; x < startx + needle_width; x++)
{
int haystack_index = y * haystack_width + x;
int needle_index = (y - starty) * needle_width + x - startx;
if (haystack[haystack_index] != needle[needle_index])
return false;
}
}
return true;
}

Instead of making two screenshots of your desktop with a time interval between them, I would take a screenshot once and cut "needle" and "haystack" from those same bitmap source. Otherwise you have the risk of a change of your desktop contents between the two moments where the screenshots are taken.
EDIT: And when your problem still occurs after that, I would try to save the image to a file and try again with that file using your debugger, giving you a reproducible situation.

First, there is a problem with the findmatch loop. You shouldn't just use the haystack image as an array, because you need to subtract needle's width and height from right and bottom respectively:
public Point? findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int y = 0; y < haystack_height - needle_height; y++)
for (int x = 0; x < haystack_width - needle_width; x++)
{
if (haystack[y * haystack_width + x] == firstpixel)
{
var matched = checkmatch(haystack, needle, x, y);
if (matched)
return (new Point(x, y));
}
}
return null;
}
That should probably solve the problem. Also, keep in mind that there might be multiple matches. For example, if "needle" is a completely white rectangle portion of a window, there will most likely be many matches in the entire screen. If this is a possibility, modify your findmatch method to continue searching for results after the first one is found:
public IEnumerable<Point> FindMatches(int[] haystack, int[] needle)
{
var firstpixel = needle[0];
for (int y = 0; y < haystack_height - needle_height; y++)
for (int x = 0; x < haystack_width - needle_width; x++)
{
if (haystack[y * haystack_width + x] == firstpixel)
{
if (checkmatch(haystack, needle, x, y))
yield return (new Point(x, y));
}
}
}
Next, you need to keep a habit of manually disposing all objects which implement IDisposable, which you have created yourself. Bitmap and Graphics are such objects, meaning that your screenshot method needs to be modified to wrap those objects in using statements:
private int[] screenshot(int x, int y, int width, int height)
{
// dispose 'bmp' after use
using (var bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb))
{
// dispose 'g' after use
using (var g = Graphics.FromImage(bmp))
{
g.CopyFromScreen(x, y, 0, 0, bmp.Size);
var bmd = bmp.LockBits(
new Rectangle(0, 0, bmp.Width, bmp.Height),
ImageLockMode.ReadOnly,
bmp.PixelFormat);
var ptr = bmd.Scan0;
// as David pointed out, "bytes" might be
// a bit misleading name for a length of
// a 32-bit int array (so I've changed it to "len")
var len = bmd.Stride * bmp.Height / 4;
var result = new int[len];
Marshal.Copy(ptr, result, 0, len);
bmp.UnlockBits(bmd);
return result;
}
}
}
The rest of the code seems ok, with the remark that it won't be very efficient for certain inputs. For example, you might have a large solid color as your desktop's background, which might result in many checkmatch calls.
If performance is of interest to you, you might want to check different ways to speed up the search (something like a modified Rabin-Karp comes to mind, but I am sure there are some existing algorithms which ensure that invalid candidates are skipped immediately).

I don't think your equations for haystack_index or needle_index are correct. It looks like you take the Scan0 offset into account when you copy the bitmap data, but you need to use the bitmap's Stride when calculating the byte position.
Also, the Format32bppArgb format uses 4 bytes per pixel. It looks like you are assuming 1 byte per pixel.
Here's the site I used to help with those equations: https://web.archive.org/web/20141229164101/http://bobpowell.net/lockingbits.aspx
Format32BppArgb: Given X and Y coordinates, the address of the first element in the
pixel is Scan0+(y * stride)+(x*4). This Points to the blue byte. The
following three bytes contain the green, red and alpha bytes.

Here are the class reference with example code that works great for my C# application for finding needle in haystack for each frame from a USB camera, in year 2018... I believe Accord is mostly a bunch of C# wrappers for fast C++ code.
Also check out the C# wrapper for Microsoft C++ DirectShow that I use to search for needle within each frame from a USB camera

Related

Image processing : How to perform this function with lockbits

I have a problem. I need to perform this function with lockbits. Please I need help.
public void xPix(Bitmap bmp, int n, Color cx, Color nx)
{
try
{
for (int y = 0; y < bmp.Height; y++)
{
for (int x = 0; x < bmp.Width; x += (n * 2))
{
cx = bmp.GetPixel(x, y);
if (x + n <= bmp.Width - 1) nx = bmp.GetPixel(x + n, y);
bmp.SetPixel(x, y, nx);
if (x + n <= bmp.Width - 1) bmp.SetPixel(x + n, y, cx);
}
}
}
catch { }
}
There were lots of things that didn't make sense to me about your code. I fixed the pieces that were preventing an image from appearing and here is the result. I will explain my changes after the code.
public void xPix(Bitmap bmp, int n, Color cx, Color nx)
{
var img = bmp.LockBits(new Rectangle(Point.Empty, bmp.Size), System.Drawing.Imaging.ImageLockMode.ReadWrite, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
byte[] bmpBytes = new byte[Math.Abs(img.Stride) * img.Height];
System.Runtime.InteropServices.Marshal.Copy(img.Scan0, bmpBytes, 0, bmpBytes.Length);
for (int y = 0; y < img.Height; y++)
{
for (int x = 0; x < img.Width; x+=n*2)
{
cx = Color.FromArgb(BitConverter.ToInt32(bmpBytes, y * Math.Abs(img.Stride) + x * 4));
if (x + n <= img.Width - 1) nx = Color.FromArgb(BitConverter.ToInt32(bmpBytes, y * Math.Abs(img.Stride) + x * 4));
BitConverter.GetBytes(nx.ToArgb()).CopyTo(bmpBytes, y * Math.Abs(img.Stride) + x * 4);
if (x + n <= img.Width - 1) BitConverter.GetBytes(cx.ToArgb()).CopyTo(bmpBytes, y * Math.Abs(img.Stride) + (x + n) * 4);
}
}
System.Runtime.InteropServices.Marshal.Copy(bmpBytes, 0, img.Scan0, bmpBytes.Length);
bmp.UnlockBits(img);
}
protected override void OnClick(EventArgs e)
{
base.OnClick(e);
Bitmap bmp = new Bitmap(#"C:\Users\bluem\Downloads\Default.png");
for (int i = 0; i < bmp.Width; i++)
{
xPix(bmp, new Random().Next(20) + 1, System.Drawing.Color.White, System.Drawing.Color.Green);
}
Canvas.Image = bmp;
}
There's no such class as LockBitmap so I replaced it with the result of a call to Bitmap.LockBits directly.
The result of LockBits does not include functions for GetPixel and SetPixel, so I did what one normally does with the result of LockBits (see https://learn.microsoft.com/en-us/dotnet/api/system.drawing.bitmap.lockbits?view=netframework-4.7.2) and copied the data into a byte array instead.
When accessing the byte data directly, some math must be done to convert the x and y coordinates into a 1-dimensional coordinate within the array of bytes, which I did.
When accessing the byte data directly under the System.Drawing.Imaging.PixelFormat.Format32bppArgb pixel format, multiple bytes must be accessed to convert between byte data and a pixel color, which I did with BitConverter.GetBytes, BitConverter.ToInt32, Color.FromArgb and Color.ToArgb.
I don't think it's a good idea to be changing the Image in the middle of painting it. You should either be drawing the image directly during the Paint event, or changing the image outside the Paint event and allowing the system to draw it. So I used the OnClick of my form to trigger the function instead.
The first random number I got was 0, so I had to add 1 to avoid an endless loop.
The cx and nx parameters never seem to be used as inputs, so I put arbitrary color values in for them. Your x and y variables were not defined/declared anywhere.
If you want faster on-image-action, you can use Marshall.Copy method with Parallel.For
Why dont use GetPixel method? Because every time you call it, your ALL image is loaded to memory. GetPixel get one pixel, and UNLOAD all image. And in every iteration, ALL image is loaded to memory (for example, if u r working on 500x500 pix image, GetPixel will load 500x500 times whole pixels to memory). When you work on images with C# (CV stuff), work on raw bytes from memory.
I will show how to use with Lockbits in Binarization because its easy to explain.
int pixelBPP = Image.GetPixelFormatSize(resultBmp.PixelFormat) / 8;
unsafe
{
BitmapData bmpData = resultBmp.LockBits(new Rectangle(0, 0, resultBmp.Width, resultBmp.Height), ImageLockMode.ReadWrite, resultBmp.PixelFormat);
byte* ptr = (byte*)bmpData.Scan0; //addres of first line
int height = resultBmp.Height;
int width = resultBmp.Width * pixelBPP;
Parallel.For(0, height, y =>
{
byte* offset = ptr + (y * bmpData.Stride); //set row
for(int x = 0; x < width; x = x + pixelBPP)
{
byte value = (offset[x] + offset[x + 1] + offset[x + 2]) / 3 > threshold ? Byte.MaxValue : Byte.MinValue;
offset[x] = value;
offset[x + 1] = value;
offset[x + 2] = value;
if (pixelBPP == 4)
{
offset[x + 3] = 255;
}
}
});
resultBmp.UnlockBits(bmpData);
}
Now, example with Marshall.copy:
BitmapData bmpData = resultBmp.LockBits(new Rectangle(0, 0, resultBmp.Width, resultBmp.Height),
ImageLockMode.ReadWrite,
resultBmp.PixelFormat
);
int bytes = bmpData.Stride * resultBmp.Height;
byte[] pixels = new byte[bytes];
Marshal.Copy(bmpData.Scan0, pixels, 0, bytes); //loading bytes to memory
int height = resultBmp.Height;
int width = resultBmp.Width;
Parallel.For(0, height - 1, y => //seting 2s and 3s
{
int offset = y * stride; //row
for (int x = 0; x < width - 1; x++)
{
int positionOfPixel = x + offset + pixelFormat; //remember about pixel format!
//do what you want with pixel
}
}
});
Marshal.Copy(pixels, 0, bmpData.Scan0, bytes); //copying bytes to bitmap
resultBmp.UnlockBits(bmpData);
Remember, when you warking with RAW bytes very important is to remember about PixelFormat. If you work on RGBA image, you need to set up every channel. (for example offset + x + pixelFormat). I showed it in Binarization example, how to deak with RGBA image with raw data. If lockbits are not fast enough, use Marshall.Copy

How can i use LockBits with a Bitmap to scan for white pixels and then write to a new bitmap all the non white pixels?

This is what i did in form1 constructor:
Bitmap bmp2 = new Bitmap(#"e:\result1001.jpg");
CropImageWhiteAreas.ImageTrim(bmp2);
bmp2.Save(#"e:\result1002.jpg");
bmp2.Dispose();
The class CropImageWhiteAreas:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
namespace Test
{
class CropImageWhiteAreas
{
public static Bitmap ImageTrim(Bitmap img)
{
//get image data
BitmapData bd = img.LockBits(new Rectangle(Point.Empty, img.Size),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
int[] rgbValues = new int[img.Height * img.Width];
Marshal.Copy(bd.Scan0, rgbValues, 0, rgbValues.Length);
img.UnlockBits(bd);
#region determine bounds
int left = bd.Width;
int top = bd.Height;
int right = 0;
int bottom = 0;
//determine top
for (int i = 0; i < rgbValues.Length; i++)
{
int color = rgbValues[i] & 0xffffff;
if (color != 0xffffff)
{
int r = i / bd.Width;
int c = i % bd.Width;
if (left > c)
{
left = c;
}
if (right < c)
{
right = c;
}
bottom = r;
top = r;
break;
}
}
//determine bottom
for (int i = rgbValues.Length - 1; i >= 0; i--)
{
int color = rgbValues[i] & 0xffffff;
if (color != 0xffffff)
{
int r = i / bd.Width;
int c = i % bd.Width;
if (left > c)
{
left = c;
}
if (right < c)
{
right = c;
}
bottom = r;
break;
}
}
if (bottom > top)
{
for (int r = top + 1; r < bottom; r++)
{
//determine left
for (int c = 0; c < left; c++)
{
int color = rgbValues[r * bd.Width + c] & 0xffffff;
if (color != 0xffffff)
{
if (left > c)
{
left = c;
break;
}
}
}
//determine right
for (int c = bd.Width - 1; c > right; c--)
{
int color = rgbValues[r * bd.Width + c] & 0xffffff;
if (color != 0xffffff)
{
if (right < c)
{
right = c;
break;
}
}
}
}
}
int width = right - left + 1;
int height = bottom - top + 1;
#endregion
//copy image data
int[] imgData = new int[width * height];
for (int r = top; r <= bottom; r++)
{
Array.Copy(rgbValues, r * bd.Width + left, imgData, (r - top) * width, width);
}
//create new image
Bitmap newImage = new Bitmap(width, height, PixelFormat.Format32bppArgb);
BitmapData nbd
= newImage.LockBits(new Rectangle(0, 0, width, height),
ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
Marshal.Copy(imgData, 0, nbd.Scan0, imgData.Length);
newImage.UnlockBits(nbd);
return newImage;
}
}
}
I also tried before it Peter solution.
In both the result is(This is a screenshot of my facebook after uploaded the image) still the white areas around:
You can the rectangle around the image i just uploaded and see what i mean by white area around.
If I understand correctly, you have found a sample code snippet that uses LockBits(), but you are not sure how it works or how to modify it to suit your specific need. So I will try to answer from that perspective.
First, a wild guess (since you didn't include the implementation of the LockBitmap class you're using in the first example): the LockBitmap class is some kind of helper class that is supposed to encapsulate the work of calling LockBits() and using the result, including providing versions of GetPixel() and SetPixel() which are presumably much faster than calling those methods on a Bitmap object directly (i.e. access the buffer obtained by calling LockBits()).
If that's the case, then modifying the first example to suit your need is probably best:
public void Change(Bitmap bmp)
{
Bitmap newBitmap = new Bitmap(bmp.Width, bmp.Height, bmp.PixelFormat);
LockBitmap source = new LockBitmap(bmp),
target = new LockBitmap(newBitmap);
source.LockBits();
target.LockBits();
Color white = Color.FromArgb(255, 255, 255, 255);
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
Color old = source.GetPixel(x, y);
if (old != white)
{
target.SetPixel(x, y, old);
}
}
}
source.UnlockBits();
target.UnlockBits();
newBitmap.Save("d:\\result.png");
}
In short: copy the current pixel value to a local variable, compare that value to the white color value, and if it is not the same, go ahead and copy the pixel value to the new bitmap.
Some variation on the second code example should work as well. The second code example does explicitly what is (I've assumed) encapsulated inside the LockBitmap class that the first code example uses. If for some reason, the first approach isn't suitable for your needs, you can follow the second example.
In that code example you provide, most of the method there is just handling the "grunt work" of locking the bitmap so that the raw data can be accessed, and then iterating through that raw data.
It computes the oRow and nRow array offsets (named for "old row" and "new row", I presume) based on the outer y loop, and then accesses individual pixel data by computing the offset within a given row based on the inner x loop.
Since you want to do essentially the same thing, but instead of converting the image to grayscale, you just want to selectively copy all non-white pixels to the new bitmap, you can (should be able to) simply modify the body of the inner x loop. For example:
byte red = oRow[x * pixelSize + 2],
green = oRow[x * pixelSize + 1],
blue = oRow[x * pixelSize];
if (red != 255 || green != 255 || blue != 255)
{
nRow[x * pixelSize + 2] = red;
nRow[x * pixelSize + 1] = green;
nRow[x * pixelSize] = blue;
}
The above would entirely replace the body of the inner x loop.
One caveat: do note that when using the LockBits() approach, knowing the pixel format of the bitmap is crucial. The example you've shown assumes the bitmaps are in 24 bpp format. If your own bitmaps are in this format, then you don't need to change anything. But if they are in a different format, you'll need to adjust the code to suit that. For example, if your bitmap is in 32 bpp format, you need to pass the correct PixelFormat value to the LockBits() method calls, and then set pixelSize to 4 instead of 3 as the code does now.
Edit:
You've indicated that you would like to crop the new image so that it is the minimize size required to contain all of the non-white pixels. Here is a version of the first example above that should accomplish that:
public void Change(Bitmap bmp)
{
LockBitmap source = new LockBitmap(bmp);
source.LockBits();
Color white = Color.FromArgb(255, 255, 255, 255);
int minX = int.MaxValue, maxX = int.MinValue,
minY = int.MaxValue, maxY = int.MinValue;
// Brute-force scan of the bitmap to find image boundary
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
if (source.GetPixel(x, y) != white)
{
if (x < minX) minX = x;
if (x > maxX) maxX = x;
if (y < minY) minY = y;
if (y > maxY) maxY = y;
}
}
}
Bitmap newBitmap = new Bitmap(maxX - minx + 1, maxY - minY + 1, bmp.PixelFormat);
LockBitmap target = new LockBitmap(newBitmap);
target.LockBits();
for (int y = 0; y < target.Height; y++)
{
for (int x = 0; x < target.Width; x++)
{
target.SetPixel(x, y, source.GetPixel(x + minX, y + minY));
}
}
source.UnlockBits();
target.UnlockBits();
newBitmap.Save("d:\\result.png");
}
This example includes an initial scan of the original bitmap, after locking it, to find the minimum and maximum coordinate values for any non-white pixel. Having done that, it uses the results of that scan to determine the dimensions of the new bitmap. When copying the pixels, it restricts the x and y loops to the dimensions of the new bitmap, adjusting the x and y values to map from the location in the new bitmap to the given pixel's original location in the old one.
Note that since the initial scan determines where the non-white pixels are, there's no need to check again when actually copying the pixels.
There are more efficient ways to scan the bitmap than the above. This version simply looks at every single pixel in the original bitmap, keeping track of the min and max values for each coordinate. I'm guessing this will be fast enough for your purposes, but if you want something faster, you can change the scan so that it scans for each min and max in sequence:
Scan each row from y of 0 to determine the first row with a non-white pixel. This is the min y value.
Scan each row from y of source.Height - 1 backwards, to find the max y value.
Having found the min and max y values, now scan the columns from x of 0 to find the min x and from source.Width - 1 backwards to find the max x.
Doing it that way involves a lot more code and is probably harder to read and understand, but would involve inspecting many fewer pixels in most cases.
Edit #2:
Here is a sample of the output of the second code example:
Note that all of the white border of the original bitmap (shown on the left side) has been cropped out, leaving only the smallest subset of the original bitmap that can contain all of the non-white pixels (shown on the right side).

Parallel.For statement return "System.InvalidOperationException" with a Bitmap Processing

Well, I have a code to apply a Rain Bow filter in "x" image, I have to do in two ways: Sequential & parallel, my sequential code is working without problems, but the parallel section doesn't work. And I have no idea, why?.
Code
public static Bitmap RainbowFilterParallel(Bitmap bmp)
{
Bitmap temp = new Bitmap(bmp.Width, bmp.Height);
int raz = bmp.Height / 4;
Parallel.For(0, bmp.Width, i =>
{
Parallel.For(0, bmp.Height, x =>
{
if (i < (raz))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R / 5, bmp.GetPixel(i, x).G, bmp.GetPixel(i, x).B));
}
else if (i < (raz * 2))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R, bmp.GetPixel(i, x).G / 5, bmp.GetPixel(i, x).B));
}
else if (i < (raz * 3))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R, bmp.GetPixel(i, x).G, bmp.GetPixel(i, x).B / 5));
}
else if (i < (raz * 4))
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R / 5, bmp.GetPixel(i, x).G, bmp.GetPixel(i, x).B / 5));
}
else
{
temp.SetPixel(i, x, Color.FromArgb(bmp.GetPixel(i, x).R / 5, bmp.GetPixel(i, x).G / 5, bmp.GetPixel(i, x).B / 5));
}
});
});
return temp;
}
Besides, In a moments the program return the same error but says "The object is already in use".
PS. I'm beginner with c#, and I Searched this topic in another post and I found nothing.
Thank you very much in advance
As commenter Ron Beyer points out, using the SetPixel() and GetPixel() methods is very slow. Each call to one of those methods involves a lot of overhead in the transition between your managed code down to the actual binary buffer that the Bitmap object represents. There are a lot of layers there, and the video driver typically gets involved which requires transitions between user and kernel level execution.
But besides being slow, these methods also make the object "busy", and so if an attempt to use the bitmap (including calling one of those methods) is made between the time one of those methods is called and when it returns (i.e. while the call is in progress), an error occurs with the exception you saw.
Since the only way that parallelizing your current code would be helpful is if these method calls could occur concurrently, and since they simply cannot, this approach isn't going to work.
On the other hand, using the LockBits() method is not only guaranteed to work, there's a very good chance that you will find the performance is so much better using LockBits() that you don't even need to parallelize the algorithm. But should you decide you do, because of the way LockBits() works — you gain access to a raw buffer of bytes that represents the bitmap image — you can easily parallelize the algorithm and take advantage of multiple CPU cores (if present).
Note that when using LockBits() you will be working with the Bitmap object at a level that you might not be accustomed to. If you are not already knowledgeable with how bitmaps really work "under the hood", you will have to familiarize yourself with the way that bitmaps are actually stored in memory. This includes understanding what the different pixel formats mean, how to interpret and modify pixels for a given format, and how a bitmap is laid out in memory (e.g. the order of rows, which can vary depending on the bitmap, as well as the "stride" of the bitmap).
These things are not terribly hard to learn, but it will require patience. It is well worth the effort though, if performance is your goal.
Parallel is hard on the singular mind. And mixing it with legacy GDI+ code can lead to strange results..
Your code has numerous issues:
You call GetPixel three times per pixel instead of once
You are accessing the pixels not horizontally as you should
You call y x and x i; the machine won't mind but us people do
You are using way too much parallelization. No use to have much more of it than you have cores. It creates overhead that is bound to eat up any gains, unless your inner loop has a really hard job to do, like millions of calculations..
But the exception you get has nothing to do with these issues. And one mistake you don't make is to access the same pixel in parallel... So why the crash?
After cleaning up the code I found that the error in the stack trace pointed to SetPixel and there to System.Drawing.Image.get_Width(). The former is obvious, the latter not part of our code..!?
So I dug into the source code at referencesource.microsoft.com and found this:
/// <include file='doc\Bitmap.uex' path='docs/doc[#for="Bitmap.SetPixel"]/*' />
/// <devdoc>
/// <para>
/// Sets the color of the specified pixel in this <see cref='System.Drawing.Bitmap'/> .
/// </para>
/// </devdoc>
public void SetPixel(int x, int y, Color color) {
if ((PixelFormat & PixelFormat.Indexed) != 0) {
throw new InvalidOperationException(SR.GetString(SR.GdiplusCannotSetPixelFromIndexedPixelFormat));
}
if (x < 0 || x >= Width) {
throw new ArgumentOutOfRangeException("x", SR.GetString(SR.ValidRangeX));
}
if (y < 0 || y >= Height) {
throw new ArgumentOutOfRangeException("y", SR.GetString(SR.ValidRangeY));
}
int status = SafeNativeMethods.Gdip.GdipBitmapSetPixel(new HandleRef(this, nativeImage), x, y, color.ToArgb());
if (status != SafeNativeMethods.Gdip.Ok)
throw SafeNativeMethods.Gdip.StatusException(status);
}
The real work is done by SafeNativeMethods.Gdip.GdipBitmapSetPixel but before that the method does a bounds check on the Bitmap's Width and Height. And while these in our case of course never change the system still won't allow accessing them in parallel and hence crashes when at some point the checks are happening interwoven. Totally uncesessary, of course, but there you go..
So GetPixel (which has the same behaviour) and SetPixel can't safely be used in a parallel processing.
Two ways out of it:
We can add locks to the code and thus make sure the checks won't happen at the 'same' time:
public static Bitmap RainbowFilterParallel(Bitmap bmp)
{
Bitmap temp = new Bitmap(bmp);
int raz = bmp.Height / 4;
int height = bmp.Height;
int width = bmp.Width;
// set a limit to parallesim
int maxCore = 7;
int blockH = height / maxCore + 1;
//lock (temp)
Parallel.For(0, maxCore, cor =>
{
//Parallel.For(0, bmp.Height, x =>
for (int yb = 0; yb < blockH; yb++)
{
int i = cor * blockH + yb;
if (i >= height) continue;
for (int x = 0; x < width; x++)
{
{
Color c;
// lock the Bitmap just for the GetPixel:
lock (temp) c = temp.GetPixel(x, i);
byte R = c.R;
byte G = c.G;
byte B = c.B;
if (i < (raz)) { R = (byte)(c.R / 5); }
else if (i < raz + raz) { G = (byte)(c.G / 5); }
else if (i < raz * 3) { B = (byte)(c.B / 5); }
else if (i < raz * 4) { R = (byte)(c.R / 5); B = (byte)(c.B / 5); }
else { G = (byte)(c.G / 5); R = (byte)(c.R / 5); }
// lock the Bitmap just for the SetPixel:
lock (temp) temp.SetPixel(x, i, Color.FromArgb(R,G,B));
};
}
};
});
return temp;
}
Note that limiting parallism is so important there is even a member in the ParallelOptions class and a parameter inParallel.For to control it! I have set the maximum core numer to 7, but this would be better:
int degreeOfParallelism = Environment.ProcessorCount - 1;
So this should save us some overhead. But still: I'd expect that to be slower than a corrected sequential method!
Instead going for a LockBits as Peter and Ron have suggested method makes things really fast (1ox) and adding parallelism potentially even faster still..
So finally to finish up this length answer, here is a Lockbits plus Limited-Parallel solution:
public static Bitmap RainbowFilterParallelLockbits(Bitmap bmp)
{
Bitmap temp = null;
temp = new Bitmap(bmp);
int raz = bmp.Height / 4;
int height = bmp.Height;
int width = bmp.Width;
Rectangle rect = new Rectangle(Point.Empty, bmp.Size);
BitmapData bmpData = temp.LockBits(rect,ImageLockMode.ReadOnly, temp.PixelFormat);
int bpp = (temp.PixelFormat == PixelFormat.Format32bppArgb) ? 4 : 3;
int size = bmpData.Stride * bmpData.Height;
byte[] data = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(bmpData.Scan0, data, 0, size);
var options = new ParallelOptions();
int maxCore = Environment.ProcessorCount - 1;
options.MaxDegreeOfParallelism = maxCore > 0 ? maxCore : 1;
Parallel.For(0, height, options, y =>
{
for (int x = 0; x < width; x++)
{
{
int index = y * bmpData.Stride + x * bpp;
if (y < (raz)) data[index + 2] = (byte) (data[index + 2] / 5);
else if (y < (raz * 2)) data[index + 1] = (byte)(data[index + 1] / 5);
else if (y < (raz * 3)) data[index ] = (byte)(data[index ] / 5);
else if (y < (raz * 4))
{ data[index + 2] = (byte)(data[index + 2] / 5);
data[index] = (byte)(data[index] / 5); }
else
{ data[index + 2] = (byte)(data[index + 2] / 5);
data[index + 1] = (byte)(data[index + 1] / 5);
data[index] = (byte)(data[index] / 5); }
};
};
});
System.Runtime.InteropServices.Marshal.Copy(data, 0, bmpData.Scan0, data.Length);
temp.UnlockBits(bmpData);
return temp;
}
While not strictly relevant I wanted to post a better faster version than any of the one's that I see given in the answers. This is the fastest way I know of to iterate through a bitmap and save the results in C#. In my work we need to go through millions of large images, this is just me grabbing the red channel and saving it for my own purposes but it should give you the idea of how to work
//Parallel Unsafe, Corrected Channel, Corrected Standard div 5x faster
private void TakeApart_Much_Faster(Bitmap processedBitmap)
{
_RedMin = byte.MaxValue;
_RedMax = byte.MinValue;
_arr = new byte[BMP.Width, BMP.Height];
long Sum = 0,
SumSq = 0;
BitmapData bitmapData = processedBitmap.LockBits(new Rectangle(0, 0, processedBitmap.Width, processedBitmap.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
//this is a much more useful datastructure than the array but it's slower to fill.
points = new ConcurrentDictionary<Point, byte>();
unsafe
{
int bytesPerPixel = Image.GetPixelFormatSize(bitmapData.PixelFormat) / 8;
int heightInPixels = bitmapData.Height;
int widthInBytes = bitmapData.Width * bytesPerPixel;
_RedMin = byte.MaxValue;
_RedMax = byte.MinValue;
byte* PtrFirstPixel = (byte*)bitmapData.Scan0;
Parallel.For(0, heightInPixels, y =>
{
//pointer to the first pixel so we don't lose track of where we are
byte* currentLine = PtrFirstPixel + (y * bitmapData.Stride);
for (int x = 0; x < widthInBytes; x = x + bytesPerPixel)
{
//0+2 is red channel
byte redPixel = currentLine[x + 2];
Interlocked.Add(ref Sum, redPixel);
Interlocked.Add(ref SumSq, redPixel * redPixel);
//divide by three since we are skipping ahead 3 at a time.
_arr[x/3, y] = redPixel;
_RedMin = redPixel < _RedMin ? _RedMin : redPixel;
_RedMax = redPixel > RedMax ? RedMax : redPixel;
}
});
_RedMean = Sum / TotalPixels;
_RedStDev = Math.Sqrt((SumSq / TotalPixels) - (_RedMean * _RedMean));
processedBitmap.UnlockBits(bitmapData);
}
}

How to convert a byte array of 19200 bytes in size where each byte represents 4 pixels (2 bits per pixel) to a bitmap arranged as 320x240 characters

I am communicating with an instrument (remote controlling it) and
one of the things I need to do is to draw the instrument screen.
In order to get the screen I issue a command and the instrument
replies with an array of bytes that represents the screen.
Below is what the instrument manual has to say about converting the response to the actual screen:
The command retrieves the framebuffer data used for the display.
It is 19200 bytes in size, 2-bits per pixel, 4 pixels per byte arranged as
320x240 characteres.
The data is sent in RLE encoded form.
To convert this data into a BMP for use in Windows, it needs to be
turned into a 4BPP. Also note that BMP files are upside down relative
to this data, i.e. the top display line is the last line in the BMP.
I managed to unpack the data, but now I am stuck on how to actually
go from the unpacked byte array to a bitmap.
My background on this is pretty close to zero and my searches
have not revealed much either.
I am looking for directions and/or articles I could use to help me
undestand how to get this done.
Any code or even pseudo code would also help. :-)
So, just to summarize it all:
How to convert a byte array of 19200 bytes in size, where
each byte represents 4 pixels (2 bits per pixel),
to a bitmap arranged as 320x240 characters.
Thanks in advance.
To do something like this, you'll want a routine like this:
Bitmap ConvertToBitmap(byte[] data, int width, int height)
{
Bitmap bm = new Bitmap(width, height, PixelFormat.Format24bppRgb);
for (int y=0; y < height; y++) {
for (int x=0; x < width; x++) {
int value = ReadPixelValue(data, x, y, width);
Color c = ConvertValToColor(value);
bm.SetPixel(x, y, c);
}
}
return bm;
}
from here, you need ReadPixelValue and ConvertValToColor.
static int ReadPixelValue(byte[] data, int x, int y, width)
{
int pixelsPerByte = 4;
// added the % pixelsPerByte to deal with width not being a multiple of pixelsPerByte,
// which won't happen in your case, but will in the general case
int bytesPerLine = width / pixelsPerByte + (width % pixelsPerByte != 0 ? 1 : 0);
int index = y * bytesPerLine + (x / pixelsPerByte);
byte b = data[index];
int pixelIndex = (x % pixelsPerByte) * 2;
// if every 4 pixels are reversed, try this:
// int pixelIndex = 8 - (x % pixelsPerByte) * 2;
return ((int b) >> pixelIndex) & 0x3;
}
Basically, I pull each set of two bits out of each byte and return it as an int.
As for converting to color that's up to you how to make heads or tail of the 4 values that come back.
Most likely you can do something like this:
static Color[] _colors = new Color[] { Color.Black, Color.Red, Color.Blue, Color.White };
static Color ConvertValToColor(int val)
{
if (val < 0 || val > _colors.Length)
throw new ArgumentOutOfRangeException("val");
return _colors[val];
}
If you have two bits per pixel, for each pixel you have 4 different possible colors. Probably the colors are indexed or just hardcoded (i.e. 0 means black, 1 white, etc).
Don't know if this is of much help ( I don't know what bitmap object you are using, but perhaps it has a regular RGB or ARGB scheme with 1 byte per channel), but in pseudo-actionscript, I think you should do something like this.
// 80 -> 320 / 4
for(var x:int = 0; x < 80; x++) {
for(var y:int = 0; y < 240; y++) {
var byteVal:int = readByte();
var px_1:int = (byteVal >> 6) & 0x03;
var px_2:int = (byteVal >> 4) & 0x03;
var px_3:int = (byteVal >> 2) & 0x03;
var px_4:int = (byteVal) & 0x03;
// map your pixel value to ARGB
px_1 = getPixelValue(px_1);
px_2 = getPixelValue(px_2);
px_3 = getPixelValue(px_3);
px_4 = getPixelValue(px_4);
// assuming setPixel(x,y,pixelValue)
setPixel((x * 4), y, px_1);
setPixel((x * 4) + 1, y, px_2);
setPixel((x * 4) + 2, y, px_3);
setPixel((x * 4) + 3, y, px_4);
}
}
function getPixelValue(idx:int):uint {
// just an example...
switch(idx) {
case 0: return 0xff000000; // black
case 1: return 0xffffffff; // white
case 2: return 0xffff0000; // red
case 3: return 0xff0000ff; // blue
}
}
The above code, suffice it to say, is just to give you an idea (hopefully!) and is based on some assumptions like how these four pixels are stored in a byte.
Hope it makes sense.
I dont know if this helps, I use this for data I got from a rare old hardware:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
byte[] imageBytes = new byte[19201];
//(Fill it with the data from the unit before doing the rest).
Bitmap bmp_workarea = new Bitmap(320, 240, System.Drawing.Imaging.PixelFormat.Format4bppIndexed);
Image newImage = Image.FromStream(new MemoryStream(imageBytes));
using (Graphics gr = Graphics.FromImage(bmp_workarea))
{
gr.DrawImage(newImage, new Rectangle(0, 0, bmp_workarea.Width, bmp_workarea.Height));
}
//now you can use newImage, for example picturebox1.image=newimage
}
}
}

C# Attempted to read or write protected memory error

Addendum: it seems to run correctly when I uncheck "optimize code" which leads me to believe it is some quirky configuration problem
Firstly I am trying to run unmanaged code. I have "allow unsafe code" checked. It is pointing to this line of code here where I am trying to read a bitmap without using the relatively slow getpixel:
byte[] buff = { scanline[xo], scanline[xo + 1], scanline[xo + 2], 0xff };
Entire snippet is below. How can I correct this problem?
private const int PIXELSIZE = 4; // Number of bytes in a pixel
BitmapData mainImageData = mainImage.LockBits(new Rectangle(0, 0, mainImage.Width, mainImage.Height), ImageLockMode.ReadOnly, mainImage.PixelFormat);
List<Point> results = new List<Point>();
foundRects = new List<Rectangle>();
for (int y = 0; y < mainImageData.Height
{
byte* scanline = (byte*)mainImageData.Scan0 + (y * mainImageData.Stride);
for (int x = 0; x < mainImageData.Width; x++)
{
int xo = x * PIXELSIZE;
byte[] buff = { scanline[xo], scanline[xo + 1],
scanline[xo + 2], 0xff };
int val = BitConverter.ToInt32(buff, 0);
// Pixle value from subimage in desktop image
if (pixels.ContainsKey(val) && NotFound(x, y))
{
Point loc = (Point)pixels[val];
int sx = x - loc.X;
int sy = y - loc.Y;
// Subimage occurs in desktop image
if (ImageThere(mainImageData, subImage, sx, sy))
{
Point p = new Point(x - loc.X, y - loc.Y);
results.Add(p);
foundRects.Add(new Rectangle(x, y, subImage.Width,
subImage.Height));
}
}
}
It's hard to say with the limited information we have, but I see a couple of obvious problems, one of which addresses your issue directly:
You are not checking the pixel format, but are assuming that it is 32bppRGB. It is likely 24bppRGB, and that would explain the error.
You are reading the RGB values incorrectly; Windows internally stores bitmaps in BGR order.
You are not calling UnlockBits at the end of the method.
What are the values xo + 1 and xo + 2 when x is mainImageData.Width - 1? They are surely walking off the reservation.
The most likely reason is that the image isn't 32bpp. it could be 24bpp, or possibly even 8 or 16 bit.
You should be getting PIXELSIZE from PixelFormat Rather than hard-coding it to 4.
Check to see that the value you are getting for Stride agrees with the Width.

Categories

Resources